502 Bad Gateway nginx/1.10.3

847 views
Skip to first unread message

Son lam Randy Nguyen

unread,
Aug 9, 2019, 5:10:27 AM8/9/19
to NetBox
Hi.. 


I have 502 Bad Gateway when i followed the great tutoriel here : 


that work very well.

as u can see :

http://prntscr.com/oqfhwc
but, i want to install the "HTTP deamon" with Nginx, but i have :


I don't know why.. i followed step by step..

Can u see my config ?

-/etc/home/netbox/netbox/netbox/netbox/configuration.py
- /etc/home/netbox/netbox/gunicorn_config.py
-/etc/nginx/sites-available/netbox
-/etc/supervisor/conf.d/netbox.conf

/home/netbox/netbox

/home/netbox/netbox/netbox

/home/netbox/netbox/netbox/netbox


I have ip adresse : 172.19.238.189 for netbox

Thanks..







/
gunicorn_config.py
netbox
netbox.conf
configuration.py

Brian Candler

unread,
Aug 9, 2019, 6:47:54 AM8/9/19
to NetBox
Based on the screenshots, it looks like you are running Netbox under Windows??

If so how (Cygwin, Microsoft's new Linux emulation, something else?)  Since you still have Linux paths like /etc, /home

Son lam Randy Nguyen

unread,
Aug 9, 2019, 6:56:06 AM8/9/19
to NetBox
No, I used Hyper-V where I installed a graphical Debian VM. Normally, I am in the same subnet as the VM so I could access from my host machine (Windows 10). And even, in the graphical debian when I open Firefox ESR, it puts me the same thing. 502 bad gateway00

Brian Candler

unread,
Aug 9, 2019, 7:04:35 AM8/9/19
to NetBox
OK: in that case, please could you copy-paste all your configs into a message as text, not as screenshots, as this is much easier to check.

The "bad gateway" error usually means that gunicorn didn't start.  Try running gunicorn directly from the command line, instead of via supervisord.  It will more likely give you an error message which explains the issue.

sudo su - www-data
cd /opt/netbox/netbox
gunicorn -c /opt/netbox/gunicorn_config.py netbox.wsgi

(That is, run it in the same way that /etc/supervisor/conf.d/netbox.conf shows)
Message has been deleted
Message has been deleted

Brian Candler

unread,
Aug 11, 2019, 5:44:34 AM8/11/19
to NetBox
> 1) i tried " su www-data" they return :
>  
> This account is currently not available.

OK, then instead do:

su - www-data -s /bin/bash

This switches user to "www-data".  Then you can run gunicorn as shown, and see if you get an error message.

Son lam Randy Nguyen

unread,
Aug 11, 2019, 2:54:34 PM8/11/19
to NetBox
1) su - www-data -s /bin/bash 
that work

but, i have this message :

gunicorn -c /home/netbox/netbox/gunicorn_config.py netbox.wsgi
[2019-08-11 20:52:20 +0000] [54482] [INFO] Starting gunicorn 19.6.0
[2019-08-11 20:52:20 +0000] [54482] [INFO] Listening at: http://192.168.94.179:8001 (54482)
[2019-08-11 20:52:20 +0000] [54482] [INFO] Using worker: sync
[2019-08-11 20:52:20 +0000] [54490] [INFO] Booting worker with pid: 54490
[2019-08-11 20:52:20 +0000] [54490] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/gunicorn/arbiter.py", line 557, in spawn_worker
    worker.init_process()
  File "/usr/lib/python2.7/dist-packages/gunicorn/workers/base.py", line 126, in init_process
    self.load_wsgi()
  File "/usr/lib/python2.7/dist-packages/gunicorn/workers/base.py", line 136, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/lib/python2.7/dist-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 65, in load
    return self.load_wsgiapp()
  File "/usr/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/lib/python2.7/dist-packages/gunicorn/util.py", line 384, in import_app
    __import__(module)
  File "/home/netbox/netbox/netbox/netbox/wsgi.py", line 3, in <module>
    from django.core.wsgi import get_wsgi_application
ImportError: No module named django.core.wsgi
[2019-08-11 20:52:20 +0000] [54490] [INFO] Worker exiting (pid: 54490)
[2019-08-11 20:52:20 +0000] [54482] [INFO] Shutting down: Master
[2019-08-11 20:52:20 +0000] [54482] [INFO] Reason: Worker failed to boot.



Brian Candler

unread,
Aug 12, 2019, 2:35:27 AM8/12/19
to NetBox
On Sunday, 11 August 2019 19:54:34 UTC+1, Son lam Randy Nguyen wrote:
ImportError: No module named django.core.wsgi

OK, there's your problem.

Most likely you have two different versions of gunicorn installed, and you're picking up the wrong one.  Try the same command but with "/usr/local/bin/gunicorn" instead of "gunicorn". 

Son lam Randy Nguyen

unread,
Aug 12, 2019, 2:51:40 AM8/12/19
to NetBox
so, i try

"  /usr/local/bin/gunicorn -c /opt/netbox/gunicorn_config.py netbox.wsgi "

but, they return nothing and also have 502 bad gateway..

i tried to restart supervisor and nginx.. same thing..

Brian Candler

unread,
Aug 12, 2019, 5:04:54 AM8/12/19
to NetBox
On Monday, 12 August 2019 07:51:40 UTC+1, Son lam Randy Nguyen wrote:
so, i try

"  /usr/local/bin/gunicorn -c /opt/netbox/gunicorn_config.py netbox.wsgi "

but, they return nothing

What do you mean, "they return nothing"?  Does it come back immediately to a shell prompt, or does it hang there?  Do you see any error?

If it just hangs there, it may simply be running.  In which case, change your supervisord config to say /usr/local/bin/gunicorn instead of gunicorn. 


and also have 502 bad gateway..

All that means most likely is that nginx is trying to proxy to a webserver which isn't running. 

Son lam Randy Nguyen

unread,
Aug 12, 2019, 5:34:25 AM8/12/19
to NetBox
when i said " They return nothing"

he is coming back immediately to the shell prompt.. 

 " If it just hangs there, it may simply be running.  In which case, change your supervisord config to say /usr/local/bin/gunicorn instead of gunicorn. "

i did it, and restart supervisor and nginx.. same thing 502 bad gateway

Brian Candler

unread,
Aug 12, 2019, 6:10:58 AM8/12/19
to NetBox
On Monday, 12 August 2019 10:34:25 UTC+1, Son lam Randy Nguyen wrote:
when i said " They return nothing"

he is coming back immediately to the shell prompt.. 


When it comes back to the shell prompt, type "echo $?" immediately afterwards.  It will show a number.  What number is it?

Then try:

strace -f /usr/local/bin/gunicorn -c /opt/netbox/gunicorn_config.py netbox.wsgi

and show us the last 100 lines or so.  (Using copy-paste please, not graphical screendump).

Son lam Randy Nguyen

unread,
Aug 19, 2019, 6:22:15 AM8/19/19
to NetBox
coming back sorry.

I tried  "strace -f /usr/local/bin/gunicorn -c /opt/netbox/gunicorn_config.py netbox.wsgi"

they return : 

root@netbox :/home/netbox# strace -f /usr/local/bin/gunicorn -c /opt/netbox/gunicorn_config.py netbox.wsgi
execve("/usr/local/bin/gunicorn", ["/usr/local/bin/gunicorn", "-c", "/opt/netbox/gunicorn_config.py", "netbox.wsgi"], [/* 19 vars */]) = -1 ENOEXEC (Exec format error)
write(2, "strace: exec: Exec format error\n", 32strace: exec: Exec format error
) = 32
exit_group(1)                           = ?
+++ exited with 1 +++

Son lam Randy Nguyen

unread,
Aug 19, 2019, 8:28:45 AM8/19/19
to NetBox
the command : 

tail -30 /var/log/nginx/error.log 

return : 

2019/08/19 14:09:28 [error] 2246#2246: *5 connect() failed (111: Connection refu                                                                                                             sed) while connecting to upstream, client: 172.17.184.97, server: 172.17.184.104                                                                                                             , request: "GET /favicon.ico HTTP/1.1", upstream: "http://172.17.184.104:8001/fa                                                                                                             vicon.ico", host: "172.17.184.104", referrer: "http://172.17.184.104/"
2019/08/19 14:11:44 [error] 2409#2409: *1 connect() failed (111: Connection refu                                                                                                             sed) while connecting to upstream, client: 172.17.184.97, server: 172.17.184.104                                                                                                             , request: "GET / HTTP/1.1", upstream: "http://172.17.184.104:8001/", host: "172                                                                                                             .17.184.104"
2019/08/19 14:11:44 [error] 2409#2409: *1 connect() failed (111: Connection refu                                                                                                             sed) while connecting to upstream, client: 172.17.184.97, server: 172.17.184.104                                                                                                             , request: "GET /favicon.ico HTTP/1.1", upstream: "http://172.17.184.104:8001/fa                                                                                                             vicon.ico", host: "172.17.184.104", referrer: "http://172.17.184.104/"
2019/08/19 14:16:26 [error] 2472#2472: *1 connect() failed (111: Connection refu                                                                                                             sed) while connecting to upstream, client: 172.17.184.97, server: 172.17.184.104                                                                                                             , request: "GET / HTTP/1.1", upstream: "http://172.17.184.104:8001/", host: "172                                                                                                             .17.184.104"
2019/08/19 14:16:26 [error] 2472#2472: *1 connect() failed (111: Connection refu                                                                                                             sed) while connecting to upstream, client: 172.17.184.97, server: 172.17.184.104                                                                                                             , request: "GET /favicon.ico HTTP/1.1", upstream: "http://172.17.184.104:8001/fa                                                                                                             vicon.ico", host: "172.17.184.104", referrer: "http://172.17.184.104/"
2019/08/19 14:19:25 [error] 2472#2472: *5 connect() failed (111: Connection refu                                                                                                             sed) while connecting to upstream, client: 172.17.184.97, server: 172.17.184.104                                                                                                             , request: "GET / HTTP/1.1", upstream: "http://172.17.184.104:8001/", host: "172                                                                                                             .17.184.104"
2019/08/19 14:19:25 [error] 2472#2472: *5 connect() failed (111: Connection refu                                                                                                             sed) while connecting to upstream, client: 172.17.184.97, server: 172.17.184.104                                                                                                             , request: "GET /favicon.ico HTTP/1.1", upstream: "http://172.17.184.104:8001/fa                                                                                                             vicon.ico", host: "172.17.184.104", referrer: "http://172.17.184.104/"

Brian Candler

unread,
Aug 19, 2019, 8:37:02 AM8/19/19
to NetBox
On Monday, 19 August 2019 11:22:15 UTC+1, Son lam Randy Nguyen wrote:
root@netbox :/home/netbox# strace -f /usr/local/bin/gunicorn -c /opt/netbox/gunicorn_config.py netbox.wsgi
execve("/usr/local/bin/gunicorn", ["/usr/local/bin/gunicorn", "-c", "/opt/netbox/gunicorn_config.py", "netbox.wsgi"], [/* 19 vars */]) = -1 ENOEXEC (Exec format error)
write(2, "strace: exec: Exec format error\n", 32strace: exec: Exec format error

Getting closer.  What does the first line of /usr/local/bin/gunicorn show?

If it looks like this:

#!/usr/bin/python3

then does /usr/bin/python3 exist with the right permissions? That is, check:

ls -l /usr/bin/python3

Son lam Randy Nguyen

unread,
Aug 19, 2019, 8:52:02 AM8/19/19
to NetBox
i'm suprised......

when i tried : 

"  cat /usr/local/bin/gunicorn "

they return nothing.......

and 

ls -l /usr/bin/python3

root@netbox:/opt/netbox/netbox# ls -l /usr/bin/python3
lrwxrwxrwx 1 root root 9 août  19 09:11 /usr/bin/python3 -> python3.5



Brian Candler

unread,
Aug 19, 2019, 9:08:50 AM8/19/19
to NetBox
On Monday, 19 August 2019 13:52:02 UTC+1, Son lam Randy Nguyen wrote:
when i tried : 

"  cat /usr/local/bin/gunicorn "

they return nothing.......


That will be why it doesn't run then :-)

ls -l /usr/local/bin/gunicorn
# just to confirm that the file exists, but is zero bytes in size.
# If so:
rm /usr/local/bin/gunicorn
pip3 install gunicorn

Then check again.

Son lam Randy Nguyen

unread,
Aug 19, 2019, 9:32:02 AM8/19/19
to NetBox
same...

i delete the gunicorn file  : rm /usr/local/bin/gunicorn

i install again : : pip3 install gunicorn

i don't know why..........



root@netbox:/home/netbox# cat /usr/local/bin/gunicorn
#!/usr/bin/python
# -*- coding: utf-8 -*-
import re
import sys

from gunicorn.app.wsgiapp import run

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(run())






Can u check again my config ??? ... 


/opt/netbox/netbox/netbox 

ALLOWED_HOSTS = ['172.17.184.98']

# PostgreSQL database configuration.
DATABASE = {
    'NAME': 'netbox',         # Database name
    'USER': 'netbox',               # PostgreSQL username
    'PASSWORD': 'Mypassword',           # PostgreSQL password
    'HOST': 'localhost',      # Database server
    'PORT': '',               # Database port (leave blank for default)
}

# This key is used for secure generation of random numbers and strings. It must never be exposed outside of this file.
# For optimal security, SECRET_KEY should be at least 50 characters in length and contain a mix of letters, numbers, and
# symbols. NetBox will not run without this defined. For more information, see
# https://docs.djangoproject.com/en/dev/ref/settings/#std:setting-SECRET_KEY
SECRET_KEY = 'P*$_iO3LV6eqAZFf!v#%57S0g^UnxEQjyK@dCMmu2(Ts)NzXc1'



_____________________________________________________

/opt/netbox/gunicorn_config.py 

command = '/usr/local/bin/gunicorn'
pythonpath = '/opt/netbox/netbox/netbox'
bind = '172.17.14.98:8001'
workers = 3
user = 'www-data'



_______________________________________________________

/etc/nginx/sites-available/netbox

erver {
    listen 80;

    server_name 172.17.184.98;

    client_max_body_size 25m;

    location /static/ {
        alias /opt/netbox/netbox/static/;
    }

    location / {
        proxy_pass http://172.17.184.98:8001;
        proxy_set_header X-Forwarded-Host $server_name;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-Proto $scheme;
        add_header P3P 'CP="ALL DSP COR PSAa PSDa OUR NOR ONL UNI COM NAV"';
    }
}
__________________



Brian Candler

unread,
Aug 19, 2019, 9:59:21 AM8/19/19
to NetBox
On Monday, 19 August 2019 14:32:02 UTC+1, Son lam Randy Nguyen wrote:
root@netbox:/home/netbox# cat /usr/local/bin/gunicorn
#!/usr/bin/python

Looks like your system is trying to use python2 instead of python3 - but Netbox doesn't support python2.

You *could* try changing that line by hand to /usr/bin/python3, and it *might* work; but really there's some underlying problem which needs fixing.

Personally I would start from scratch, and I would *not* install supervisord.  supervisord depends on python2, so it means your system ends up with both python2 and python3 installed simultaneously.  Starting from a fresh Ubuntu 18.04, you'd have only python3.

You can find instructions for running Netbox under systemd instead of supervisord here.

Good luck!

Son lam Randy Nguyen

unread,
Aug 19, 2019, 10:03:23 AM8/19/19
to NetBox
Ok

that night, i'll try to install with a fresh ubuntu 18.04....

and not install Supervisor.. i'll see u tomorrow... 

thanks.

Son lam Randy Nguyen

unread,
Aug 20, 2019, 5:18:08 AM8/20/19
to NetBox
New error.

Now, it's ok.. maybe.. except CSS... i don't know why...


i did : python3 manage.py collectstatic --no-input the folder "static" is located " /opt/netbox/netbox/static" the display is : http://prntscr.com/ov0gb8 now, to reach netbox, the link is : " http://172.19.238.189:8000"

display :


My configuration :
/opt/netbox/netbox/netbox/configuration.py

ALLOWED_HOSTS = ['172.19.238.189']

# PostgreSQL database configuration.
DATABASE = {
    'NAME': 'netbox',         # Database name
    'USER': 'netbox',               # PostgreSQL username
    'PASSWORD': 'xxxxxxx',           # PostgreSQL password
    'HOST': 'localhost',      # Database server
    'PORT': '',               # Database port (leave blank for default)
}

# This key is used for secure generation of random numbers and strings. It must never be exposed outside of this fil$
# For optimal security, SECRET_KEY should be at least 50 characters in length and contain a mix of letters, numbers,$
# symbols. NetBox will not run without this defined. For more information, see
SECRET_KEY = 'O0gJ%sD5GzBy@_T8)tE7lof#uFw-9!SKM&A^b4ZI3W(6$imq=h'

/etc/nginx/sites-availables/netbox

server { listen 80; server_name 172.19.238.189; client_max_body_size 25m; location /static/ { alias /opt/netbox/netbox/static/; } location / { proxy_pass http://172.19.238.189:8001; proxy_set_header X-Forwarded-Host $server_name; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-Proto $scheme; add_header P3P 'CP="ALL DSP COR PSAa PSDa OUR NOR ONL UNI COM NAV"'; } }
/opt/netbox/gunicorn_config.py


command = '/usr/local/bin/gunicorn' pythonpath = '/opt/netbox/netbox/netbox' bind = '172.19.238.189:8000' workers = 3 user = 'www-data'
/etc/systemd/system/netbox.service

[Unit] Description=Netbox - IP address management (IPAM) and data center infrastructure management (DCIM) tool After=network-online.target Wants=network-online.target Conflicts=shutdown.target [Service] User=www-data ExecStart=/usr/local/bin/gunicorn -c /opt/netbox/gunicorn_config.py netbox.wsgi WorkingDirectory=/opt/netbox/netbox Restart=on-failure RestartSec=5 [Install] WantedBy=multi-user.target I'm close... just need CSS....



Son lam Randy Nguyen

unread,
Aug 20, 2019, 5:20:11 AM8/20/19
to NetBox
i forgot..

http://172.19.238.189

display : 

Welcome to nginx!

If you see this page, the nginx web server is successfully installed and working. Further configuration is required.

For online documentation and support please refer to nginx.org.
Commercial support is available at nginx.com.

Thank you for using nginx.


Brian Candler

unread,
Aug 20, 2019, 9:05:53 AM8/20/19
to NetBox
On Tuesday, 20 August 2019 10:18:08 UTC+1, Son lam Randy Nguyen wrote:
now, to reach netbox, the link is : " http://172.19.238.189:8000"


That's the wrong URL - you are talking directly to gunicorn there, and gunicorn doesn't know how to serve static files (i.e. the CSS).  Hence you don't get formatting.

You should talk to nginx on http://172.19.238.189/.  If configured correctly, nginx will serve the static files directly, and proxy the dynamic content to gunicorn.

 

location / { proxy_pass http://172.19.238.189:8001;

That's not going to work, because...


command = '/usr/local/bin/gunicorn' pythonpath = '/opt/netbox/netbox/netbox' bind = '172.19.238.189:8000'

... you've configured gunicorn to listen on port 8000.


That is: for gunicorn use


and for nginx use


Then try again.  (The exact values of "127.0.0.1" are correct, as long as nginx and gunicorn are both running on the same host.  But both nginx and gunicorn need to have the same).

However, you should have got an error instead of "Welcome to nginx!".  This suggests you have some spurious files in /etc/nginx/sites-enabled/

The instructions tell you to delete /etc/nginx/sites-enabled/default.  Did you do so?

The instructions tell you create a symlink for the netbox configuration.  Did you do so?

The instructions give you the exact commands to use:

# cd /etc/nginx/sites-enabled/ # rm default # ln -s /etc/nginx/sites-available/netbox

Confirm you have done this by:

cd /etc/nginx/sites-enabled
ls -l

You should see "netbox" pointing to /etc/nginx/sites-available/netbox, and no other files.
Reply all
Reply to author
Forward
0 new messages