Externalize headless service in kubernetes

[george...@gmail.com]

Hello everybody,

I'm reading the documentation and exploring headless services from kubernetes.
However, I cannot find an official way on externalizing the FQDN names of the headless service to the outside world. I found a couple of links but they are either third party tools or incubation projects, is there a best way to do this with kubernetes 1.6?

Tools that I found so far:
https://github.com/kubernetes-incubator/external-dns
https://github.com/wearemolecule/route53-kubernetes

I'm deploying kubernetes on AWS.

Thanks!
George

[Tim Hockin]

In short, you can't. That would require multiple IPs which are
allocated and deallocated and retargetted as pods come and go. This
is not supported for now. Maybe you can talk more about what you're
trying to do?

> --
> You received this message because you are subscribed to the Google Groups "Kubernetes user discussion and Q&A" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-use...@googlegroups.com.
> To post to this group, send email to kubernet...@googlegroups.com.
> Visit this group at https://groups.google.com/group/kubernetes-users.
> For more options, visit https://groups.google.com/d/optout.

[george...@gmail.com]

Hi Tim,

Thanks for the quick replay.
Essentially, I have something like the headless service described here https://kubernetes.io/docs/tutorials/stateful-application/zookeeper/ deployed on kubernetes which is running on AWS. I wanna write data to those zookeeper pods from a AWS machine deployed in the same subnet and availability zone as the kubernetes worker nodes. So what I need is some sort of mechanism to map requests from the outside world to one zookeeper node like this one zk-0.zk-headless.default.svc.cluster.local:2181. Is that possible?

Thanks!

On Monday, May 22, 2017 at 5:53:23 PM UTC+2, Tim Hockin wrote:
> In short, you can't. That would require multiple IPs which are
> allocated and deallocated and retargetted as pods come and go. This
> is not supported for now. Maybe you can talk more about what you're
> trying to do?
>

[Matthias Rampke]

In our clusters, we have solved this in the following (not elegant, very roundabout) way:

On each node (as a daemonset) we run a BIND pod that grabs port 53 in the host network namespace. We needed this anyway because some applications did not deal well with the latency of going over the network for every DNS resolution, which is totally the applications' fault. This BIND has a _forward zone_ configured for the cluster domain, forwarding all queries under this domain to the Cluster IP of the kubedns service.

Because this BIND does caching, and no client calls kubedns directly, we have removed dnsmasq from the kubedns pod, but that is entirely optional.

Finally, we have configured out non-Kubernetes internal resolvers to have a subset of Kubernetes nodes in a forward zone for the cluster domain. This way, a query from outside of Kubernetes is forwarded first to the per-node caches and from there (if not cached) to kubedns.

This only works because we control the whole DNS chain; if you are using the AWS resolvers you may have a harder time.

/MR 

[Tim Hockin]

Can you set up a proper Service for one or more nodes (one if you care
which one, more if you don't)?

On Mon, May 22, 2017 at 9:14 AM, <george...@gmail.com> wrote:
> On Monday, May 22, 2017 at 5:53:23 PM UTC+2, Tim Hockin wrote:

>> In short, you can't. That would require multiple IPs which are
>> allocated and deallocated and retargetted as pods come and go. This
>> is not supported for now. Maybe you can talk more about what you're
>> trying to do?
>>

>> On Mon, May 22, 2017 at 8:43 AM wrote:
>> > Hello everybody,
>> >
>> > I'm reading the documentation and exploring headless services from kubernetes.
>> > However, I cannot find an official way on externalizing the FQDN names of the headless service to the outside world. I found a couple of links but they are either third party tools or incubation projects, is there a best way to do this with kubernetes 1.6?
>> >
>> > Tools that I found so far:
>> > https://github.com/kubernetes-incubator/external-dns
>> > https://github.com/wearemolecule/route53-kubernetes
>> >
>> > I'm deploying kubernetes on AWS.
>> >
>> > Thanks!
>> > George
>> >
>> > --
>> > You received this message because you are subscribed to the Google Groups "Kubernetes user discussion and Q&A" group.
>> > To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-use...@googlegroups.com.
>> > To post to this group, send email to kubernet...@googlegroups.com.
>> > Visit this group at https://groups.google.com/group/kubernetes-users.
>> > For more options, visit https://groups.google.com/d/optout.
>

> Hi Tim,
>
> Thanks for the quick replay.
> Essentially, I have something like the headless service described here https://kubernetes.io/docs/tutorials/stateful-application/zookeeper/ deployed on kubernetes which is running on AWS. I wanna write data to those zookeeper pods from a AWS machine deployed in the same subnet and availability zone as the kubernetes worker nodes. So what I need is some sort of mechanism to map requests from the outside world to one zookeeper node like this one zk-0.zk-headless.default.svc.cluster.local:2181. Is that possible?
>
>
> Thanks!
>

[george...@gmail.com]

Hello guys,

@Matthias thanks for your suggestion, your setup is a bit to complex for me right now.
@Tim so you suggest instead of having a headless service which is pointing to a stateful set with 3 replicas like in this example https://kubernetes.io/docs/tutorials/stateful-application/zookeeper/, to have one service per zookeeper node? And how will this help with accessing the service from outside?

Thanks,
George

On Monday, May 22, 2017 at 9:31:14 PM UTC+2, Tim Hockin wrote:
> Can you set up a proper Service for one or more nodes (one if you care
> which one, more if you don't)?
>

[Rodrigo Campos]

On Tuesday, May 23, 2017, <george...@gmail.com> wrote:

Hello guys,

@Matthias thanks for your suggestion, your setup is a bit to complex for me right now.
@Tim so you suggest instead of having a headless service which is pointing to a stateful set with 3 replicas like in this example https://kubernetes.io/docs/tutorials/stateful-application/zookeeper/, to have one service per zookeeper node? And how will this help with accessing the service from outside?

Yes, with services you can access from outside too. If you need to connect to specific nodes, then one service per zookeeper node should do the trick.

[Tim Hockin]

I was suggesting a headless Service to provide the DNS that you
generally want in a StatefulSet *and* and normal Service of
type=LoadBalancer, which can expose one or more of your ZK instances
to the outside world.

Sorry, I said "node" before when I meant zk instance. :)