Invalid HTTP_HOST header when website being accessed by public IP

[Kasper Laudrup]

Hi fellow Django users,

I have succesfully deployed a small Django site with uwsgi and Nginx to
a virtual server running in Amazons cloud (AWS).

I have also succesusfully set up email so I will get an email everytime
an error occurs. Quite useful.

Now, my problem is, that lately I have been receiving quite a lot of
emails since there seems to be some bots (or whatever) that tries to
access my website through its public IP, causing "Invalid HTTP_HOST
header" errors.

I could quite easily (and I have actually already written the code for
that) dynamically figure out my servers public IP and add that to the
ALLOWED_HOSTS setting in settings.py, but I'm not certain that is the
correct solution?

I would think it's an error to access my website through its IP (in the
HTTP Host header), but it's quite anoying to get emails everytime some
bot, crawler or whatever attempts to do that.

Anyone having faced this issue before? Would it be correct simply to add
the public IP to the list of allowed hosts, or is there a better
solution? I definitely still want to get emails when any other error occurs.

Thanks a lot!

Kind regards,
Kasper Laudrup

[Tomasz Knapik]

I don't think that's wise to set allowed hosts to a host you don't want your application to be accessed by. Django documentation shows you how you can mute the errors - https://docs.djangoproject.com/en/2.0/topics/logging/#django-security. It's not technically an error of your application.

'handlers': {
    'null': {
        'class': 'logging.NullHandler',
    },
},
'loggers': {
    'django.security.DisallowedHost': {
        'handlers': ['null'],
        'propagate': False,
    },
},

Maybe you could restrict host headers at the nginx layer, but I don't think it's worth your effort... Maybe you should use some smarter solution for receiving errors like Sentry where you only get notified once about an error and you can mute them instead of relying on each error occurrence triggering an email :P

Also you could integrate more into AWS and use their load balancing service where you should be able to set routing based on host header.

[Kasper Laudrup]

Hi Tomasz,

On 2018-07-01 19:10, Tomasz Knapik wrote:
> I don't think that's wise to set allowed hosts to a host you don't want
> your application to be accessed by.

I agree completely, which is why I asked the question. Thanks a lot.

> Django documentation shows you how
> you can mute the errors -
> https://docs.djangoproject.com/en/2.0/topics/logging/#django-security.
> It's not technically an error of your application.

That's very helpful. Again thanks a lot. I mostly wanted to know what
was the right way to handle this and I think just ignoring errors like
this is probably the best way in my use case.

> Maybe you could restrict host headers at the nginx layer, but I don't
> think it's worth your effort... Maybe you should use some smarter
> solution for receiving errors like Sentry where you only get notified
> once about an error and you can mute them instead of relying on each
> error occurrence triggering an email :P
>
> Also you could integrate more into AWS and use their load balancing
> service where you should be able to set routing based on host header.
>

I think all of that is probably very much overkill considering that this
is just a private homepage that at most has a couple of visitors a day,
but thanks for the pointers.

Very much appreciated.

Kind regards,

Kasper Laudrup

[Melvyn Sopacua]

On zondag 1 juli 2018 19:10:15 CEST Tomasz Knapik wrote:

 

> Maybe you could restrict host headers at the nginx layer, but I don't

> think it's worth your effort...

 

If you think of it like that it seems like a lot of work. But if you simply setup a default server that redirects to the actual Django server with correct hostname, then all you need is 2 server blocks: 1 default, 1 with correct `server_name`.

 

See Request Processing for more background information and tricks.

--

Melvyn Sopacua

[Kasper Laudrup]

Hi Melvyn,

On 2018-07-02 11:32, Melvyn Sopacua wrote:
> On zondag 1 juli 2018 19:10:15 CEST Tomasz Knapik wrote:
>
> > Maybe you could restrict host headers at the nginx layer, but I don't
>
> > think it's worth your effort...
>
> If you think of it like that it seems like a lot of work. But if you
> simply setup a default server that redirects to the actual Django server
> with correct hostname, then all you need is 2 server blocks: 1 default,
> 1 with correct `server_name`.
>

You are correct. That was actually fairly easy to fix by changing the
nginx configuration.

I didn't do exactly as you mentioned since I use HTTPS (with a redirect
for HTTP) managed by letsencrypt.

Instead I added the following to my HTTPS server section:

if ($host != my-website.org {
return 404;
}

Seems to solve my problem just fine. Letsencrypts certbot had already
done something similar for the HTTP section redirect.

Thanks a lot for the input.

Kind regards,

Kasper Laudrup

[Melvyn Sopacua]

On maandag 2 juli 2018 17:25:20 CEST Kasper Laudrup wrote:

> Instead I added the following to my HTTPS server section:
>
> if ($host != my-website.org {
> return 404;
> }
>
> Seems to solve my problem just fine. Letsencrypts certbot had already
> done something similar for the HTTP section redirect.

The only reason to set it up like that for HTTPS is that it's possible the SNI
name differs from the HTTP Host header. For HTTP redirects it makes no sense:
the HTTP header is in plain text and is used to determine the server block to
pick. So putting an if statement there, is just doing it again, on every
request, because electrons are cheap. Save the electrons!

Anyhow - instead of return 404, I would do:

return 301 https://$server_name$request_uri

How I normally set things up:

server {
listen 443 default_server ssl http2;
server_name localhost;

return 301 https://djangoserver.example.com$request_uri
}

server {
listen 443;
server_name djangoserver.example.com;

# ... django setup
}
--
Melvyn Sopacua

[Kasper Laudrup]

Hej again Melvyn,

On 07/03/2018 12:13 AM, Melvyn Sopacua wrote:
>
> The only reason to set it up like that for HTTPS is that it's possible the SNI
> name differs from the HTTP Host header. For HTTP redirects it makes no sense:
> the HTTP header is in plain text and is used to determine the server block to
> pick. So putting an if statement there, is just doing it again, on every
> request, because electrons are cheap. Save the electrons!
>

Thanks for your explanation.

So, if I understand you correctly, it does make sense to explicitly test
for the host header in a HTTPS request, like I have done, but not in the
setup I got from letsencrypt where HTTP traffic is redirected to HTTPS
if the HOST headers matches?

> Anyhow - instead of return 404, I would do:
>
> return 301 https://$server_name$request_uri
>

I guess I could do that as well, but then I guess whatever is trying to
crawl, exploit whatever my host would get redirected to the actual site,
which I find unnecessary. Right?

> How I normally set things up:
>
> server {
> listen 443 default_server ssl http2;
> server_name localhost;
>
> return 301 https://djangoserver.example.com$request_uri
> }
>

Wouldn't this require having at least a self signed certifate for this
server section, even if it's just used to redirect? Admittedly, I
haven't tried it.

Thanks a lot for your help.

Kind regards,

Kasper Laudrup