Help getting K8s Gerrit setup using ISTIO in AWS EKS

[bar...@telinformix.com]

I got a site up and running using INGRESS and ingress-nginx but using that method I couldn't get the real source address to appear in Gerrits sshd_log though it was showing in the httpd_log.

So I thought I'd try using ISTIO as the ingress but I'm having even more issues.  I can get to the site on port 80 but 443 and 29418 don't seem to work.  I can understand why 443 doesn't work as that is mapping to 8443 but nothing is listening in my cluster on that port so I'm obviously missing something but 29418 should work however the health checks are failing as are the ones for Port 15012

The only changes I made to the istio profile is in the overlays section :

overlays:
       overlays:
          - kind: Service
            name: istio-ingressgateway
            patches:
              #- path: spec.loadBalancerIP
                # TO_BE_CHANGED: Change IP
                #value: xxx.xxx.xxx.xxx
              - path: spec.loadBalancerSourceRanges
                # TO_BE_CHANGED: Change IP-Range to whitelist
                value:
                  - 0.0.0.0/0
              - path: metadata.annotations
                # TO_BE_CHANGED: Annotations to be set in the service, e.g. to
                # configure automated DNS and certificate management in Gardener
                # value:
                #   dns.gardener.cloud/dnsnames: '*.example.com'
                #   dns.gardener.cloud/class: garden
                #   dns.gardener.cloud/ttl: "600"
                #   cert.gardener.cloud/commonName: '*.example.com'
                #   cert.gardener.cloud/purpose: managed
                #   cert.gardener.cloud/secretname: tls-secret
                value:
                  service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
                  service.beta.kubernetes.io/aws-load-balancer-backend-protocol: ssl
                  service.beta.kubernetes.io/aws-load-balancer-scheme: "internal"
                  service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "arn:aws:acm:xxxxx"
                  service.beta.kubernetes.io/aws-load-balancer-ssl-negotiation-policy: ELBSecurityPolicy-TLS-1-2-2017-01
                  service.beta.kubernetes.io/aws-load-balancer-ssl-ports: https
                  service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: "ip"
                  service.beta.kubernetes.io/aws-load-balancer-target-group-attributes: preserve_client_ip.enabled=true

                  external-dns.alpha.kubernetes.io/hostname: gerrit.bgarrett.sw.cirrus.com

The gerrit operator is labeled as per the instructions and the gerrit pod has been restarted and has the istio-proxy container running in it.

setting service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: "ip" results in target groups that point to the istioigressgateways on the configured ports rather than the nodes and kube-proxy ports.

To be honest I'm not even sure where to begin getting this working.  I think the 2 main things I'm missing are the SSL part, what should be listening on port 8443 in the EKS cluster, and why is 29418 not connecting ?

[Thomas Dräbing]

On Tue, 15 Jul 2025 at 17:27, bar...@telinformix.com <bar...@telinformix.com> wrote:

I got a site up and running using INGRESS and ingress-nginx but using that method I couldn't get the real source address to appear in Gerrits sshd_log though it was showing in the httpd_log.

That is indeed not possible. The same holds true for Istio unfortunately. Using TCP this would only work using the PROXY protocol which is as of yet unsupported by Gerrit.

Could you provide the Gerrit Cluster resource that you are using? At least the ingress configuration? Did you configure the Gerrit Operator to use Istio? The operator only supports a single Ingress Provider at a time. If everything is configured correctly, you should find VirtualService, Gateway and ServiceEntry resources in your cluster, which were created by the operator. This might help to see what is going wrong. A helpful tool to identify issues with the istio service mesh is Kiali, which allows you to visualize the service mesh and see the logs and configuration for each connection in a structured way.

HTH,

Thomas

 

--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/repo-discuss/ae4f82df-2e6f-40c8-bdfd-4e3eb50470c7n%40googlegroups.com.

[bar...@telinformix.com]

On Thursday, July 17, 2025 at 8:19:28 AM UTC+1 Thomas Dräbing wrote:

On Tue, 15 Jul 2025 at 17:27, bar...@telinformix.com <bar...@telinformix.com> wrote:

I got a site up and running using INGRESS and ingress-nginx but using that method I couldn't get the real source address to appear in Gerrits sshd_log though it was showing in the httpd_log.

That is indeed not possible. The same holds true for Istio unfortunately. Using TCP this would only work using the PROXY protocol which is as of yet unsupported by Gerrit.

oh well.  

I ran:
helm upgrade --install -n gerrit-operator gerrit-operator helm-charts/gerrit-operator/ --set=ingress.type=ISTIO  
kubectl label namespace gerrit istio-injection=enabled

kubectl describe ns gerrit
    Name:         gerrit
    Labels:       istio-injection=enabled
                kubernetes.io/metadata.name=gerrit
    Annotations:  <none>
    Status:       Active
    No resource quota.
    No LimitRange resource.

and I've attached the resources you listed.

I saw something about Kiali when I was searching docs but I'm afraid ISTIO seems awfully complex to me, it might take a while to work out how to install it :)

[Thomas Dräbing]

Port 443 should be enabled with your configuration, but looking at your Gateway resource it wasn't. That might be an issue with the Operator. Could you check whether the GerritNetwork resource was updated to enable TLS?

To enable port 29418 (SSH), set `spec.ingress.ssh.enabled = true`. Seems that this is missing in the API reference :-(.

The HTTS port in the Gateway, i.e. the port Istio will listen to for ingress requests, is currently hard-coded to 443. This would have to be made configurable to support port 8443, or you would have to install network related resources yourself (using ingress.type=NONE in the Gerrit Operator).

 

HTH,

Thomas

 

--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/repo-discuss/ae4f82df-2e6f-40c8-bdfd-4e3eb50470c7n%40googlegroups.com.

--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/repo-discuss/e1358a01-baf1-41e7-9590-ad0f5a529297n%40googlegroups.com.

[Barney Garrett]

apiVersion: gerritoperator.google.com/v1beta15
kind: GerritNetwork
metadata:
  annotations:
    javaoperatorsdk.io/previous: 1dc1dcbe-8dfb-4e05-8d88-3a3838ccc6df,74030702
  creationTimestamp: "2025-06-23T11:24:21Z"
  generation: 3
  name: gerrit-gerrit-network
  namespace: gerrit
  ownerReferences:
  - apiVersion: gerritoperator.google.com/v1beta15
    kind: GerritCluster
    name: gerrit
    uid: 84641f73-e013-44bb-aa6a-9e4a496420c5
  resourceVersion: "74039022"
  uid: 3b49247d-6aed-454c-9838-13eaf13d73bb
spec:
  ingress:
    ambassador:
      createHost: false
    enabled: true
    host: gerrit.xxxxxx
    istio:
      gatewaySelector:
        istio: ingressgateway
    pathPrefix: ""
    ssh:
      enabled: false
    tls:
      enabled: true
  primaryGerrit:
    httpPort: 80
    name: gerrit
    sshPort: 29418

 

The HTTS port in the Gateway, i.e. the port Istio will listen to for ingress requests, is currently hard-coded to 443. This would have to be made configurable to support port 8443, or you would have to install network related resources yourself (using ingress.type=NONE in the Gerrit Operator).

I'm not exactly wanting to use 8443 ... it is what the provided istio profile set up, I've attached what I'm currently using, which is basically the default but with my best guesses at annotations for the AWS nlb.