Challenge returns 404

[Mark Ehler]

Seeking help on my deployment.  You'll find all deployment yamls in raw below the reproduction steps

I'm having some issues with cert-manager completing a challenge request while using cert-manager and nginx.

I've created a K8s cluster using Azure kubernetes service and taken the following steps

1 - create cluster
az aks create -g devrg -n devaks \
    --node-count 1 \
    --service-principal <sp-id> \
    --client-secret <sp-key> \
    --enable-managed-identity \
    --generate-ssh-keys \
    --enable-cluster-autoscaler \
    --min-count 1 \
    --max-count 4 > dev_cluster_data.js

2 - assign public ip using the service cluster
az network public-ip create --resource-group MC_devrg_devsaks_eastus --name publicip --sku Standard \
--allocation-method static --query publicIp.ipAddress -o tsv

3 - deploy nginx using Helm
helm install nginx-ingress ingress-nginx/ingress-nginx \
--namespace dev \
--set controller.replicaCount=2 \
--set controller.service.loadBalancerIP="172.172.208.197" \
--set controller.service.annotations."service.beta.kubernetes.io/azure-dns-label-name"="publicip"

4 - cert-manager using Helm

kubectl label namespace "dev" cert-manager.io/disable-validation=false

helm install cert-manager --namespace dev --set installCRDs=true jetstack/cert-manager

5 - Apply issuer.yaml cert.yaml
I apply the attached yaml files to the dev namespace and time and again get the following error when describing the challenge object.
-error-
Status:
  Presented:   true
  Processing:  true
  Reason:      Waiting for HTTP-01 challenge propagation: wrong status code '404', expected '200'
  State:       pending

I've tried multiple times and can't figure out why this would be so.  maybe there's something wrong with my approach?  Applying the (not yet shared) ingress.yaml deployment.yaml and services.yaml doesn't change the error.



files

---

apiVersion: cert-manager.io/v1

kind: Certificate

metadata:

  name: example-app

  namespace: dev

spec:

  dnsNames:

    - dev.pratus.ninja

  secretName: example-app

  issuerRef:

    name: letsencrypt-cluster-issuer

    kind: ClusterIssuer

---

apiVersion: cert-manager.io/v1

kind: ClusterIssuer

metadata:

  name: letsencrypt-cluster-issuer

spec:

  acme:

    server: https://acme-v02.api.letsencrypt.org/directory

    email: de...@disastertech.com

    privateKeySecretRef:

      name: letsencrypt-cluster-issuer

    solvers:

    - http01:

        ingress:

          class: nginx

          podTemplate:

            spec:

              nodeSelector:

                "kubernetes.io/os": linux