nginx ingress controller configuration on Kubernetes vagrant

[Mugdho Debnath]

I was working on setting up an on premises kubernetes cluster using vagrant. The cluster is the default multi VM cluster which I set up following the docs at https://coreos.com/kubernetes/docs/latest/kubernetes-on-vagrant.html that uses CoreOS.

Following the instructions at https://medium.com/@rothgar/exposing-services-using-ingress-with-on-prem-kubernetes-clusters-f413d87b6d34#.cclkvdn9s I set up a pod and its associated service. So far so good and I was able to have the pod and Service running. The details of both are as below.

$ kube-ctl describe pod microbot-1276362308-i65nt|grep -i node
Node:           172.17.4.201/172.17.4.201

$ kubectl describe svc microbot
Name:                   microbot
Namespace:              default
Labels:                 run=microbot
Selector:               run=microbot
Type:                   NodePort
IP:                     10.3.0.29
Port:                   <unset> 80/TCP
NodePort:               <unset> 31643/TCP
Endpoints:              10.2.43.6:80
Session Affinity:       None
No events.

Running curl on from the host to 172.17.4.201:31643 gets me the required result.

To this I added the nginx replication controller from https://github.com/kubernetes/contrib using the example mentioned here https://github.com/kubernetes/contrib/tree/master/ingress/controllers/nginx/examples/default. The sole change was the backend service the yaml to be microbot.

Lastly I added the ingress to the cluster

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
    name: microbot-ingress
spec:
    rules:
    - host: microbot
      http:
          paths:
          - path: /
            backend:
                serviceName: microbot
                servicePort: 80

Watching the logs on the nginx controller I can see it reload and if I get the conf, the microbot details are reflected. However when I try to call the url to host microbot I get a cannot connect to server error

$ curl -L --resolve microbot:80:172.17.4.201 http://microbot/
curl: (7) Failed to connect to microbot port 80: Connection refused

In the example that I had linked to, the traefik controller is used and it gives the same result. I came across a similar question here - http://stackoverflow.com/questions/35532801/kubernetes-ingress-controller-on-vagrant but the OP doesnt mention anything about how he was able to fix it.

So is there something very obvious that I may be missing from the steps with regards to setup or the usage of the IP because it seems like it should work.

Any help would be appreciated

[Yuriy Filonov]

Hi Mugdho,

I am having an exact same issue here. I also tried exposing microbot deployment on another port, but with no success.

Were you able to resolve this issue?

Cheers,

Yuriy.

[Mugdho Debnath]

Hi Yurily,

Sorry I just noticed your question. Google had filed this in a weird folder for some reason. Anyway, since its almost a month, let me know if you are still facing this issue.

Mugdho

--
You received this message because you are subscribed to a topic in the Google Groups "Kubernetes user discussion and Q&A" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kubernetes-users/ClDVAU5vKM4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kubernetes-use...@googlegroups.com.
To post to this group, send email to kubernet...@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

[Yuriy Filonov]

Thanks for responding Mugdho :)

Actually, I was not able to get microbot example going locally since then (switched to other tasks).

Did you have any success with this? Can you post your solution here, if any?

To unsubscribe from this group and all its topics, send an email to kubernetes-users+unsubscribe@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.

Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to a topic in the Google Groups "Kubernetes user discussion and Q&A" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kubernetes-users/ClDVAU5vKM4/unsubscribe.

To unsubscribe from this group and all its topics, send an email to kubernetes-users+unsubscribe@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.

[bain...@gmail.com]

I am also hitting this same issue so would be interested to know the solution if either of you have figured this out.

Thanks
Jack

[Yuriy Filonov]

Unfortunately, I was not able to get an ingress controller working on local setup.

Now I am using NodePort locally and load balancers in cloud.

[Mugdho Debnath]

This works pretty well if you move from using CoreOS to Ubuntu (or any other linux distro) VMs. From the bit of digging that I did, it seems that nginx / traefix is not opening the required port on CoreOS for some reason.

On Wed, Dec 14, 2016 at 3:19 PM Yuriy Filonov <yuriy....@gmail.com> wrote:

Unfortunately, I was not able to get an ingress controller working on local setup.

Now I am using NodePort locally and load balancers in cloud.

On Wed, Dec 14, 2016 at 10:57 PM, <bain...@gmail.com> wrote:

I am also hitting this same issue so would be interested to know the solution if either of you have figured this out.

Thanks
Jack

--
You received this message because you are subscribed to a topic in the Google Groups "Kubernetes user discussion and Q&A" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kubernetes-users/ClDVAU5vKM4/unsubscribe.

To unsubscribe from this group and all its topics, send an email to kubernetes-use...@googlegroups.com.
To post to this group, send email to kubernet...@googlegroups.com.

Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "Kubernetes user discussion and Q&A" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kubernetes-users/ClDVAU5vKM4/unsubscribe.

To unsubscribe from this group and all its topics, send an email to kubernetes-use...@googlegroups.com.
To post to this group, send email to kubernet...@googlegroups.com.

[Brandon Philips]

On Thu, Dec 15, 2016 at 3:30 PM Mugdho Debnath <mug...@gmail.com> wrote:

This works pretty well if you move from using CoreOS to Ubuntu (or any other linux distro) VMs. From the bit of digging that I did, it seems that nginx / traefix is not opening the required port on CoreOS for some reason.

Odd. Adding a few folks from the CoreOS team to see why Container Linux isn't working with this ingress setup.

Thank You,

Brandon

 

On Wed, Dec 14, 2016 at 3:19 PM Yuriy Filonov <yuriy....@gmail.com> wrote:

Unfortunately, I was not able to get an ingress controller working on local setup.

Now I am using NodePort locally and load balancers in cloud.

On Wed, Dec 14, 2016 at 10:57 PM, <bain...@gmail.com> wrote:

I am also hitting this same issue so would be interested to know the solution if either of you have figured this out.

Thanks
Jack

--
You received this message because you are subscribed to a topic in the Google Groups "Kubernetes user discussion and Q&A" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kubernetes-users/ClDVAU5vKM4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kubernetes-use...@googlegroups.com.
To post to this group, send email to kubernet...@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "Kubernetes user discussion and Q&A" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kubernetes-users/ClDVAU5vKM4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kubernetes-use...@googlegroups.com.
To post to this group, send email to kubernet...@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "Kubernetes user discussion and Q&A" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-use...@googlegroups.com.

[Aaron Levy]

This very likely has to do with HostPorts not working when CNI network plugins are used. In the case of the CoreOS installation instructions, we do make use of CNI (for flannel/calico).

See this issue for more detail: https://github.com/kubernetes/kubernetes/issues/23920

Essentially what is happening is that the ingress controller pod wants to use port 80 on the host network, but this does not work currently when using CNI. I'm not super caught up on the resolutions in the CNI plugins themselves (should have more detail in the link above). But possible workaround would be to have the ingress controller use `hostNetwork:true`, but that means all containers in that pod will attempt to bind to host ports. Another option might be to put a nodeport service in front of the ingress controller, and access it through the assigned nodeport (default: 30000-32000).