DNS service discovery on GKE with kube-airflow

[yazgoo]

Hi,

I'm using airflow on GKE (kubernetes 1.4), more precisely this yaml :

https://github.com/mumoshu/kube-airflow/blob/master/airflow.all.yaml

My problem is that the web pod needs access to http logs from the worker pod, which is equivalent to do, from the web pod, a:

curl http://worker-2088828153-64hkb:8793/log/dataflow_bigquery/dataflow_batch/2017-01-17T16:03:43.291968

Which results in

curl: (6) Could not resolve host: worker-2088828153-64hkb

The worker pod registers itself via its hostname (via rabbitmq AFAIK).

Is there a way to have worker-2088828153-64hkb hostname available in kube-DNS ?

If not, what DNS should I use ? How would I apply it as hostname to the worker container ?

Is there a way to list DNS entries ?

I tried using hostNetwork: true with no success since there is service discovery done via postgres, rabbitmq hostnames and that becomes unavailable then:

https://github.com/mumoshu/kube-airflow/blob/master/script/entrypoint.sh

Thanks

[Tim Hockin]

We don't do DNS for pods except for StatefulSets because it can change
rapidly, and DNS is just no good at that. You can set up a Service to
select all of the workers, and if you specify "None" as `clusterIP`
you won't get a VIP, just a bunch of A records.

> --
> You received this message because you are subscribed to the Google Groups
> "Kubernetes user discussion and Q&A" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kubernetes-use...@googlegroups.com.
> To post to this group, send email to kubernet...@googlegroups.com.
> Visit this group at https://groups.google.com/group/kubernetes-users.
> For more options, visit https://groups.google.com/d/optout.

[yazgoo]

Hi,

I've added,

apiVersion: v1
kind: Service
metadata:
  name: worker
spec:
  clusterIP: None
  type: NodePort
  selector:
    app: worker
  ports:
    - name: worker
      protocol: TCP
      port: 8793
      targetPort: worker
      nodePort: 32082

This does not seem to work, the issue being that the worker still registers itself with it's hostname to the web UI.

Any suggestion ?

[Tim Hockin]

Its hostname doesn't work unless you follow the section "A Records and
hostname based on Pod’s hostname and subdomain fields" in
https://kubernetes.io/docs/admin/dns/

On Wed, Jan 18, 2017 at 12:24 AM, yazgoo <olivier.a...@teads.tv> wrote:
> Hi,
>

> I've added,
>
> apiVersion: v1
> kind: Service
> metadata:
> name: worker
> spec:
> clusterIP: None
> type: NodePort
> selector:
> app: worker
> ports:
> - name: worker
> protocol: TCP
> port: 8793

> targetPort: flower

[yazgoo]

Thanks,

I've added:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: worker
spec:
  replicas: 1
  template:
    metadata:
      labels:
        name: worker
        app: airflow
        tier: worker
    spec:
      hostname: worker-1

Once I have done that, how can I address the worker Pod from the web Pod ?

What hostname should/can I use ?

Must I also set a subdomain ?

Thanks

[Tim Hockin]

You must spec hostname and subdomain. subdomain must be the name of a
headless service. That way the pod's notion of hostname will match
the DNS name assigned.

[Youssef Essa]

Hey Yazgoo,
Are you solved this problem? I face the same issue especially when scale up number of workers.