MetalLB assigns an IP Address to a Nginx Ingress Controller but the site cannot be reached

[Leon Damjanov]

I am trying to create a demo ecommerce application with the purpose of showing the capabilities of distributed tracing and OpenTelemetry. My idea is to create a simple Ecommerce store where the user can click different buttons and these buttons will create traces with the help of OpenTelemetry and will show these traces on a Jaeger dashboard. At first I had only a frontend built with browser js but found out that OpenTelemetry doesnt fully support browser javascript at the moment. I split my code into a frontend and backend which runs on NodeJS. My program is deployed in a minikube cluster and I couldnt figure out why my frontend and backend couldnt connect. I read somewhere that a Nginx Ingress would be a solution for this which is what I've tried. But Minikube wouldnt expose the Ingress and the external ip was stuck at "<pending>". I installed MetalLB onto my cluster and I got it to give the nginx ingress an external ip. But now when I try to ping or open that ip in my browser, I get a "ERR_CONNECTION_TIMED_OUT" error.

I ran the logs on both the ingress and metallb controller pods and here are the results from each:

kubectl logs nginx-ingress-ingress-nginx-controller-7487b4c6b-g67bg

------------------------------------------------------------------------------- NGINX Ingress controller Release: v1.9.4 Build: 846d251814a09d8a5d8d28e2e604bfc7749bcb49 Repository: https://github.com/kubernetes/ingress-nginx nginx version: nginx/1.21.6 ------------------------------------------------------------------------------- W1210 21:46:45.432805 7 client_config.go:618] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work. I1210 21:46:45.432971 7 main.go:205] "Creating API client" host="https://10.96.0.1:443" I1210 21:46:45.438367 7 main.go:249] "Running in Kubernetes cluster" major="1" minor="27" git="v1.27.4" state="clean" commit="fa3d7990104d7c1f16943a67f11b154b71f6a132" platform="linux/amd64" I1210 21:46:45.595799 7 main.go:101] "SSL fake certificate created" file="/etc/ingress-controller/ssl/default-fake-certificate.pem" I1210 21:46:45.612325 7 ssl.go:536] "loading tls certificate" path="/usr/local/certificates/cert" key="/usr/local/certificates/key" I1210 21:46:45.622290 7 nginx.go:260] "Starting NGINX Ingress controller" I1210 21:46:45.627301 7 event.go:298] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"default", Name:"nginx-ingress-ingress-nginx-controller", UID:"2593209a-fe5e-42a0-bf40-931bf4a39e3e", APIVersion:"v1", ResourceVersion:"1074", FieldPath:""}): type: 'Normal' reason: 'CREATE' ConfigMap default/nginx-ingress-ingress-nginx-controller I1210 21:46:46.824382 7 nginx.go:303] "Starting NGINX process" I1210 21:46:46.824484 7 leaderelection.go:245] attempting to acquire leader lease default/nginx-ingress-ingress-nginx-leader... I1210 21:46:46.825377 7 nginx.go:323] "Starting validation webhook" address=":8443" certPath="/usr/local/certificates/cert" keyPath="/usr/local/certificates/key" I1210 21:46:46.825570 7 controller.go:190] "Configuration changes detected, backend reload required" I1210 21:46:46.832670 7 leaderelection.go:255] successfully acquired lease default/nginx-ingress-ingress-nginx-leader I1210 21:46:46.832714 7 status.go:84] "New leader elected" identity="nginx-ingress-ingress-nginx-controller-7487b4c6b-g67bg" I1210 21:46:46.888273 7 controller.go:210] "Backend successfully reloaded" I1210 21:46:46.888416 7 controller.go:221] "Initial sync, sleeping for 1 second" I1210 21:46:46.888517 7 event.go:298] Event(v1.ObjectReference{Kind:"Pod", Namespace:"default", Name:"nginx-ingress-ingress-nginx-controller-7487b4c6b-g67bg", UID:"76c5124c-7ced-43d8-a212-6703ce16ac66", APIVersion:"v1", ResourceVersion:"1106", FieldPath:""}): type: 'Normal' reason: 'RELOAD' NGINX reload triggered due to a change in configuration W1210 21:47:01.420857 7 controller.go:1214] Service "default/simple-ecommerce-frontend" does not have any active Endpoint. W1210 21:47:01.420928 7 controller.go:1214] Service "default/simple-ecommerce-backend" does not have any active Endpoint. I1210 21:47:01.531744 7 admission.go:149] processed ingress via admission controller {testedIngressLength:1 testedIngressTime:0.111s renderingIngressLength:1 renderingIngressTime:0s admissionTime:25.5kBs testedConfigurationSize:0.111} I1210 21:47:01.531795 7 main.go:107] "successfully validated configuration, accepting" ingress="default/simple-ecommerce-ingress" I1210 21:47:01.573962 7 store.go:440] "Found valid IngressClass" ingress="default/simple-ecommerce-ingress" ingressclass="nginx" I1210 21:47:01.574282 7 event.go:298] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"simple-ecommerce-ingress", UID:"7bdaab16-1096-4cd5-89b1-0086382c1056", APIVersion:"networking.k8s.io/v1", ResourceVersion:"1251", FieldPath:""}): type: 'Normal' reason: 'Sync' Scheduled for sync W1210 21:47:02.650140 7 controller.go:1214] Service "default/simple-ecommerce-frontend" does not have any active Endpoint. W1210 21:47:02.650190 7 controller.go:1214] Service "default/simple-ecommerce-backend" does not have any active Endpoint. I1210 21:47:02.650273 7 controller.go:190] "Configuration changes detected, backend reload required" I1210 21:47:02.755896 7 controller.go:210] "Backend successfully reloaded" I1210 21:47:02.756160 7 event.go:298] Event(v1.ObjectReference{Kind:"Pod", Namespace:"default", Name:"nginx-ingress-ingress-nginx-controller-7487b4c6b-g67bg", UID:"76c5124c-7ced-43d8-a212-6703ce16ac66", APIVersion:"v1", ResourceVersion:"1106", FieldPath:""}): type: 'Normal' reason: 'RELOAD' NGINX reload triggered due to a change in configuration W1210 21:47:30.239289 7 controller.go:1214] Service "default/simple-ecommerce-frontend" does not have any active Endpoint. W1210 21:47:33.572800 7 controller.go:1214] Service "default/simple-ecommerce-frontend" does not have any active Endpoint. W1210 21:47:36.906494 7 controller.go:1214] Service "default/simple-ecommerce-frontend" does not have any active Endpoint. I1210 21:47:46.840376 7 status.go:304] "updating Ingress status" namespace="default" ingress="simple-ecommerce-ingress" currentValue=null newValue=[{"ip":"192.168.0.200"}] I1210 21:47:48.528095 7 event.go:298] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"simple-ecommerce-ingress", UID:"7bdaab16-1096-4cd5-89b1-0086382c1056", APIVersion:"networking.k8s.io/v1", ResourceVersion:"1319", FieldPath:""}): type: 'Normal' reason: 'Sync' Scheduled for sync W1210 21:47:48.528239 7 controller.go:1214] Service "default/simple-ecommerce-frontend" does not have any active Endpoint.

kubectl logs -n metallb-system controller-565ccc769f-npv8v https://prnt.sc/UlExXefNpL4W http://prntscr.com/RqzTJ4wN9gLM

Here are my current yaml files for the ingress, metallb, frontend and backend:

metallb-config.yaml:

apiVersion: metallb.io/v1beta1 kind: IPAddressPool metadata: name: first-pool namespace: metallb-system spec: addresses: - 192.168.0.200-192.168.0.210

l2advertisement-config.yaml:

apiVersion: metallb.io/v1beta1 kind: L2Advertisement metadata: name: second-pool namespace: metallb-system spec: ipAddressPools: - first-pool

ingress.yaml:

apiVersion: networking.k8s.io/v1 kind: Ingress metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"networking.k8s.io/v1","kind":"Ingress","metadata":{"annotations":{},"name":"simple-ecommerce-ingress","namespace":"default"},"spec":{"ingressClassName":"nginx","rules":[{"host":"simple-ecommerce-service.test","http":{"paths":[{"backend":{"service":{"name":"simple-ecommerce-frontend","port":{"number":80}}},"path":"/","pathType":"Prefix"},{"backend":{"service":{"name":"simple-ecommerce-backend","port":{"number":3000}}},"path":"/api","pathType":"Prefix"}]}}]}} creationTimestamp: "2023-12-10T13:22:53Z" generation: 1 name: simple-ecommerce-ingress namespace: default resourceVersion: "1922" uid: f6e86ff5-631c-4c70-b14b-3db7788f1442 spec: ingressClassName: nginx rules: - host: simple-ecommerce-service.test http: paths: - backend: service: name: simple-ecommerce-frontend port: number: 80 path: / pathType: Prefix - backend: service: name: simple-ecommerce-backend port: number: 3000 path: /api pathType: Prefix

frontend-service.yaml:

apiVersion: v1 kind: Service metadata: name: simple-ecommerce-frontend spec: selector: app: simple-ecommerce-frontend ports: - protocol: TCP port: 80 targetPort: 80 type: ClusterIP

backend-service.yaml:

apiVersion: v1 kind: Service metadata: name: simple-ecommerce-backend spec: selector: app: simple-ecommerce-backend ports: - protocol: TCP port: 3000 targetPort: 3000 type: ClusterIP

I've tried disabling my Firewall. I have "192.168.0.200 simple-ecommerce-service.test" added in my Hosts file.

Tried to install a different version of metallb but did not work. I dont understand why the ip assigned to the ingress does not work. The frontend and backend worked as separate services.

[Leon Damjanov]

EDIT: I posted the issue on Stackoverflow as well: https://stackoverflow.com/questions/77636803/metallb-assigns-an-ip-address-to-a-nginx-ingress-controller-but-the-site-cannot

[li li]

The latest version of metallb installed, looks at metallb's pod logs and shows that the ipaddress has been assigned properly, and ingress-controoler has obtained the ipaddress written to the ipaddresspool configuration, but the machine cannot ping through the IP address. It seems that I am not the only one who has encountered this problem. All articles on the Internet are relatively old. It seems that configmap was used for configuration before version 0.13, but after version 0.13, ipaddresspool is used for configuration. Now, there's no solution to this problem, which is if you're using the latest version of metallb, it's not going to work on a bare metal machine