HELP! AWX on EKS with ALBs

[Chris Grigor]

Hello all, 

I have recently tried to deploy using the AWX Operator on EKS, I'have been battling to understand what is going wrong.

1. first deploy the aws-load-balancer-controller

helm install aws-load-balancer-controller eks/aws-load-balancer-controller -n kube-system --set clusterName=myclustername

2. deploy AWX operator with helm
helm search repo awx-operator
NAME                            CHART VERSION   APP VERSION     DESCRIPTION
awx-operator/awx-operator       0.17.1          0.17.1          A Helm chart for the AWX Operator

$ helm install my-awx-operator awx-operator/awx-operator
NAME: my-awx-operator
LAST DEPLOYED: Fri Jun 10 11:28:24 2022
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Helm Chart 0.22.0

3. Create my manifest and deploy to awx namepsace.
(domain name mydomain is replaced with my real domain in my deployment)

---
apiVersion: awx.ansible.com/v1beta1
kind: AWX
metadata:
  name: awx
spec:
  service_type: LoadBalancer
  service_annotations: |
    environment: mgmt
  service_labels: |
    environment: mgmt
  hostname: awx.mydomain.com
  ingress_annotations: |
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/healthcheck-path: "/"
    alb.ingress.kubernetes.io/group.name: "default-alb"
    alb.ingress.kubernetes.io/ssl-policy: "ELBSecurityPolicy-FS-1-2-Res-2020-10"
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-1:12345678910:certificate/12345678-1234-xxxxxxxxxxxxxxxxxxxx
    alb.ingress.kubernetes.io/load-balancer-attributes: "routing.http.x_amzn_tls_version_and_cipher_suite.enabled=true"
    alb.ingress.kubernetes.io/healthcheck-protocol: HTTP
    alb.ingress.kubernetes.io/healthcheck-port: traffic-port
    alb.ingress.kubernetes.io/healthcheck-interval-seconds: '15'
    alb.ingress.kubernetes.io/healthcheck-timeout-seconds: '5'
    alb.ingress.kubernetes.io/success-codes: '200'
    alb.ingress.kubernetes.io/healthy-threshold-count: '2'
    alb.ingress.kubernetes.io/unhealthy-threshold-count: '2'
    kubernetes.io/ingress.class: alb
    external-dns.alpha.kubernetes.io/hostname: awx.mydomain.com
  ingress_type: Ingress
  ingress_path: "/*"
  loadbalancer_protocol: https
  loadbalancer_port: 443
  route_tls_termination_mechanism: Edge
  projects_persistence: true
  projects_storage_access_mode: ReadWriteOnce
  # web_extra_volume_mounts: |
  #   - name: static-data
  #     mountPath: /var/lib/projects
  # extra_volumes: |
  #   - name: static-data
  #     persistentVolumeClaim:
  #       claimName: awx-projects-claim
# Web POd Limits
  web_resource_requirements:
    requests:
      cpu: 400m
      memory: 2Gi
    limits:
      cpu: 1000m
      memory: 4Gi
  task_resource_requirements:
    requests:
      cpu: 250m
      memory: 1Gi
    limits:
      cpu: 500m
      memory: 2Gi
  ee_resource_requirements:
    requests:
      cpu: 250m
      memory: 1Gi
    limits:
      cpu: 500m
      memory: 2Gi
---
apiVersion: v1
kind: Secret
metadata:
  name: awx-admin-password
  namespace: awx
stringData:
  password: thisismysuperlongpassword

The result is that it creates

(load balancer external-ip names are changed in this example)

~/D/w/ansible-eks-cluster $ kc get ep -n awx
NAME                                              ENDPOINTS           AGE
awx                                               10.141.1.148:8080   47m
awx-operator-controller-manager-metrics-service   10.141.1.171:8443   119m
awx-postgres                                      10.141.1.11:5432    84m
awx-service                                       10.141.1.148:8052   83m

~/D/w/ansible-eks-cluster $ kc get svc -n awx

awx                                               LoadBalancer   172.20.66.47    xxxxxxxxxxx.us-east-1.elb.amazonaws.com    8080:32541/TCP   47m
awx-operator-controller-manager-metrics-service   ClusterIP      172.20.36.148   <none>                                                                    8443/TCP         120m
awx-postgres                                      ClusterIP      None            <none>                                                                    5432/TCP         84m
awx-service                                       LoadBalancer   172.20.72.206   xxxxxxxxxxxxxbbbbbbb.us-east-1.elb.amazonaws.com   443:30050/TCP    84m

~/D/w/ansible-eks-cluster $ kc get pods -n awx

NAME                                               READY   STATUS    RESTARTS   AGE
awx-69d8d7594-s9dc7                                4/4     Running   0          76m
awx-operator-controller-manager-7db4d49d8b-zkfbc   2/2     Running   0          122m
awx-postgres-0                                     1/1     Running   0          80m

~/D/w/ansible-eks-cluster $ kc get ingress -n awx
NAME          CLASS    HOSTS              ADDRESS   PORTS   AGE
awx-ingress   <none>   awx.mydomain.com             80      4h34m

What this does do is automatically create a classic load balancer with a listener on port 443 forwarding to the node port 30050

When I got the external address https://awx.mydomain.com I get an nginx landing page (see screenshot attached)

Can anyone see what I have done wrong? I'm just lost right now, I have been over this so many times.

Thanks in advance!

Kind regards

Chris