Http status code for default-backend-service

[Michele Orsi]

Hi,

we are working on creating a cluster on kubernetes with bare metal. We use the ingress resource to map our services and we use the nginx ingress controller from the contrib project to manage them:

https://github.com/kubernetes/contrib/tree/master/ingress/controllers/nginx

We are now in the phase of checking the nginx behaviour. We noticed that when it doesn’t found a pod to redirect a request to it just serve a 404 error due to the 404-server called.

https://github.com/kubernetes/contrib/tree/master/404-server

Now from our perspective this is a big problem that should be addressed with a 5xx error, so we were thinking about re-implementing the 404-error with a 503-server that just serve 503 http status. 

Generally speaking, 404 is a valid REST response code (for missing resource) while 503 is a proper server error. If it serves 404 our infrastructure will not be able to properly monitor when a service is not available.

Are we missing something in this way? Do you think that this could affect the cluster in (another) a bad way? 

Or are we just not addressing in the right way the problem?

Any help on this is really appreciated,

Thanks so much,

Michele 

[Rajat Chopra]

That is correct. Backend not available should mean 503 and not 404, even though 503 only suggests temporary unavailability.

/Rajat

> Listed on SIX Swiss Exchange
>
> Our websites: lastminute.com | bravofly.com | volagratis.com | rumbo.es | jetcost.com | group.lastminute.com
>
> This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden and could be a crime.
>
> --
> You received this message because you are subscribed to the Google Groups "kubernetes-sig-network" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-ne...@googlegroups.com.
> To post to this group, send email to kubernetes-...@googlegroups.com.
> Visit this group at https://groups.google.com/group/kubernetes-sig-network.
> For more options, visit https://groups.google.com/d/optout.

[Prashanth B]

On Sat, May 14, 2016 at 6:15 AM, Rajat Chopra <rajat...@movingwheel.com> wrote:

That is correct. Backend not available should mean 503 and not 404, even though 503 only suggests temporary unavailability.

/Rajat

On May 14, 2016 3:36 AM, "Michele Orsi" <michel...@lastminute.com> wrote:
>
> Hi,
>
> we are working on creating a cluster on kubernetes with bare metal. We use the ingress resource to map our services and we use the nginx ingress controller from the contrib project to manage them:
> https://github.com/kubernetes/contrib/tree/master/ingress/controllers/nginx
>
> We are now in the phase of checking the nginx behaviour. We noticed that when it doesn’t found a pod to redirect a request to it just serve a 404 error due to the 404-server called.
> https://github.com/kubernetes/contrib/tree/master/404-server
>

Do you mean your ingress has no rule, or all endpoints that service a given ingress rule are dead?

i.e this is a 404: http://google.com/lkjasdklfj

but google.com when there are no servers to serve google.com is a 5xx

[Michele Orsi]

That's a good question Prashanth!

I had in mind the situation where there are no pods to serve our request, but actually there are also those other situations.

So here could be a recap with meaningful status code:

• no ingress mapping available for specific request (404 error)
• no service that maps specific ingress rule (5xx error)
• no pods that are able to reply to that specific ingress/service (5xx error)

I don't see here:

https://github.com/kubernetes/contrib/blob/master/ingress/controllers/nginx/nginx/main.go

the possibility to pass any other error page apart from the 404 error. Probably it could be a development we should do on our own and then open a MR.

Do you agree?

Thanks,

Michele Orsi 

[Prashanth B]

On Mon, May 16, 2016 at 1:53 AM, Michele Orsi <michel...@lastminute.com> wrote:

That's a good question Prashanth!

I had in mind the situation where there are no pods to serve our request, but actually there are also those other situations.

So here could be a recap with meaningful status code:

• no ingress mapping available for specific request (404 error)
• no service that maps specific ingress rule (5xx error)
• no pods that are able to reply to that specific ingress/service (5xx error)

So if the controller is returning a 404 for the 2 situations that you've indicated as 5xx, that's a bug. The second one is gray, because I can see how Service not found is similar to page not found (in both cases the controller doesn't know who is at fault, all it knows is something that should exist, doesn't). The third seems like a clear 5xx, because we can assume that if there are no endpoints it's because the pods are faiilng a readiness check, and *not* because the pods are "not found". 

 

I don't see here:

https://github.com/kubernetes/contrib/blob/master/ingress/controllers/nginx/nginx/main.go

the possibility to pass any other error page apart from the 404 error.

So I guess in addition to fixing the 5xx return code you also want a custom 5xx page? please open a bug and we can discuss on it. 

 

Probably it could be a development we should do on our own and then open a MR.

 

Sure, this would work too instead of the bug. 

 

Do you agree?

Thanks,

Michele Orsi 

[Michele Orsi]

Just performed all the tests on our bare-metal cluster and here are the results:

- no ingress mapping available for specific request -> 404 error

- when the service mapped is not there -> 404 error

- when there are no pods that satisfy specific service -> 404 error

We are now investigating this solution:

https://github.com/aledbf/contrib/tree/nginx-debug-server/Ingress/images/nginx-error-server

[Michele Orsi]

Interesting behaviour just discovered.

If you deploy the nginx-error-server (https://hub.docker.com/r/aledbf/nginx-error-server/) as the --default-backend-service in the nginx ingress controller everything works as expected:

- when you drop the service -> 503 error from the first request

- when you remove all the pods -> 502 for some seconds then 503

- if you remove ingress (rule) -> 404

That's great!!!!!

So the nginx ingress controller works as expected, the problem was that the default http backend returns always 404.

Michele 

[Prashanth B]

On Thu, May 19, 2016 at 5:56 AM, Michele Orsi <michel...@lastminute.com> wrote:

Interesting behaviour just discovered.

If you deploy the nginx-error-server (https://hub.docker.com/r/aledbf/nginx-error-server/) as the --default-backend-service in the nginx ingress controller everything works as expected:

- when you drop the service -> 503 error from the first request

- when you remove all the pods -> 502 for some seconds then 503

- if you remove ingress (rule) -> 404

Great please send a pr to update the docs/examples that confused you

That's great!!!!!

So the nginx ingress controller works as expected, the problem was that the default http backend returns always 404.

Yes if you were using https://github.com/kubernetes/contrib/tree/master/404-server, it does what it says on the tin :) 

 

Michele 

[Dirty Chat]

Guys who can help me with my site i get error default backend - 404 when i try to access any page.