netbox status 203

[Andrew Meyer]

Hello,

I'm trying to run on netbox on RHEL 8.4.1.  SELinux enforcing with other customizations.  I have the correct SELinux booleans allowed through.  However I am getting the following error and I can't figure out why.

 sudo systemctl start netbox netbox-rq;sudo systemctl status netbox netbox-rq -l

● netbox.service - NetBox WSGI Service

   Loaded: loaded (/etc/systemd/system/netbox.service; enabled; vendor preset: disabled)

   Active: activating (auto-restart) (Result: exit-code) since Fri 2021-11-05 19:41:38 UTC; 5s ago

     Docs: https://netbox.readthedocs.io/en/stable/

  Process: 19401 ExecStart=/opt/dcim/netbox/venv/bin/gunicorn --pid /var/tmp/netbox.pid --pythonpath /opt/dcim/netbox/netbox --config /opt/dcim/netbox/gunicorn.py netbox.wsgi (code=exited, status=203/EXEC)

 Main PID: 19401 (code=exited, status=203/EXEC)

    Tasks: 0 (limit: 49311)

   Memory: 0B

   CGroup: /system.slice/netbox.service

● netbox-rq.service - NetBox Request Queue Worker

   Loaded: loaded (/etc/systemd/system/netbox-rq.service; enabled; vendor preset: disabled)

   Active: activating (auto-restart) (Result: exit-code) since Fri 2021-11-05 19:41:44 UTC; 28ms ago

     Docs: https://netbox.readthedocs.io/en/stable/

  Process: 19405 ExecStart=/opt/dcim/netbox/venv/bin/python3 /opt/dcim/netbox/netbox/manage.py rqworker high default low (code=exited, status=203/EXEC)

 Main PID: 19405 (code=exited, status=203/EXEC)

Nov 05 19:41:44 d1xnetbxapp01.paneracloud.com systemd[1]: netbox-rq.service: Main process exited, code=exited, status=203/EXEC

Nov 05 19:41:44 d1xnetbxapp01.paneracloud.com systemd[1]: netbox-rq.service: Failed with result 'exit-code'.

Nov  5 19:43:09 server1 systemd[1]: netbox.service: Service RestartSec=30s expired, scheduling restart.

Nov  5 19:43:09 server1 systemd[1]: netbox.service: Scheduled restart job, restart counter is at 6.

Nov  5 19:43:09 server1 systemd[1]: Stopped NetBox WSGI Service.

Nov  5 19:43:09 server1 systemd[1]: Started NetBox WSGI Service.

Nov  5 19:43:09 server1 systemd[19442]: netbox.service: Failed to execute command: Permission denied

Nov  5 19:43:09 server1 systemd[19442]: netbox.service: Failed at step EXEC spawning /opt/dcim/netbox/venv/bin/gunicorn: Permission denied

Nov  5 19:43:09 server1 systemd[1]: netbox.service: Main process exited, code=exited, status=203/EXEC

Nov  5 19:43:09 server1 systemd[1]: netbox.service: Failed with result 'exit-code'.

Nov  5 19:43:14 server1 systemd[1]: netbox-rq.service: Service RestartSec=30s expired, scheduling restart.

Nov  5 19:43:14 server1 systemd[1]: netbox-rq.service: Scheduled restart job, restart counter is at 501.

Nov  5 19:43:14 server1 systemd[1]: Stopped NetBox Request Queue Worker.

Nov  5 19:43:14 server1 systemd[1]: Started NetBox Request Queue Worker.

Nov  5 19:43:14 server1 systemd[19446]: netbox-rq.service: Failed to execute command: Permission denied

Nov  5 19:43:14 server1 systemd[19446]: netbox-rq.service: Failed at step EXEC spawning /opt/dcim/netbox/venv/bin/python3: Permission denied

Nov  5 19:43:14 server1 systemd[1]: netbox-rq.service: Main process exited, code=exited, status=203/EXEC

Nov  5 19:43:14 server1 systemd[1]: netbox-rq.service: Failed with result 'exit-code'.

 ~]$ ls -lahZ /opt/dcim/netbox-3.0.8/

total 84K

drwxrwxr-x.  9 root netbox system_u:object_r:usr_t:s0                  4.0K Nov  5 15:42 .

drwxr-xr-x.  3 root root   system_u:object_r:unlabeled_t:s0              68 Nov  5 15:10 ..

-rw-rw-r--.  1 root netbox system_u:object_r:httpd_sys_rw_content_t:s0 2.9K Oct 20 13:49 base_requirements.txt

-rw-rw-r--.  1 root netbox system_u:object_r:httpd_sys_rw_content_t:s0  117 Oct 20 13:49 CHANGELOG.md

drwxrwxr-x.  2 root netbox system_u:object_r:httpd_sys_rw_content_t:s0  139 Oct 20 13:49 contrib

-rw-rw-r--.  1 root netbox system_u:object_r:httpd_sys_rw_content_t:s0 9.1K Oct 20 13:49 CONTRIBUTING.md

drwxrwxr-x. 15 root netbox system_u:object_r:httpd_sys_rw_content_t:s0 4.0K Oct 20 13:49 docs

-rw-rw-r--.  1 root netbox system_u:object_r:httpd_sys_rw_content_t:s0  222 Oct 20 13:49 .gitattributes

drwxrwxr-x.  4 root netbox system_u:object_r:httpd_sys_rw_content_t:s0   77 Oct 20 13:49 .github

-rw-rw-r--.  1 root netbox system_u:object_r:httpd_sys_rw_content_t:s0  458 Oct 20 13:49 .gitignore

-rw-------.  1 root netbox system_u:object_r:httpd_sys_rw_content_t:s0  486 Nov  5 15:29 gunicorn.py

-rw-rw-r--.  1 root netbox system_u:object_r:httpd_sys_rw_content_t:s0  10K Oct 20 13:49 LICENSE.txt

-rw-------.  1 root netbox system_u:object_r:httpd_sys_rw_content_t:s0   23 Nov  5 15:22 local_requirements.txt

-rw-rw-r--.  1 root netbox system_u:object_r:httpd_sys_rw_content_t:s0 5.1K Oct 20 13:49 mkdocs.yml

drwxrwxr-x. 17 root netbox system_u:object_r:httpd_sys_rw_content_t:s0 4.0K Nov  5 15:24 netbox

-rw-rw-r--.  1 root netbox system_u:object_r:httpd_sys_rw_content_t:s0   72 Oct 20 13:49 NOTICE

drwx------.  2 root netbox system_u:object_r:httpd_sys_rw_content_t:s0   37 Nov  5 15:42 __pycache__

-rw-rw-r--.  1 root netbox system_u:object_r:httpd_sys_rw_content_t:s0 3.7K Oct 20 13:49 README.md

-rw-rw-r--.  1 root netbox system_u:object_r:httpd_sys_rw_content_t:s0  614 Oct 20 13:49 requirements.txt

drwxrwxr-x.  3 root netbox system_u:object_r:httpd_sys_rw_content_t:s0   48 Oct 20 13:49 scripts

-rwxrwxr-x.  1 root netbox system_u:object_r:httpd_sys_rw_content_t:s0 3.8K Oct 20 13:49 upgrade.sh

drwx------.  5 root netbox system_u:object_r:httpd_sys_rw_content_t:s0   74 Nov  5 15:23 venv

[Unit]

Description=NetBox WSGI Service

Documentation=https://netbox.readthedocs.io/en/stable/

After=network-online.target

Wants=network-online.target

[Service]

Type=simple

User=netbox

Group=netbox

PIDFile=/var/tmp/netbox.pid

WorkingDirectory=/opt/dcim/netbox/netbox

ExecStart=/opt/dcim/netbox/venv/bin/gunicorn --pid /var/tmp/netbox.pid --pythonpath /opt/dcim/netbox/netbox --config /opt/dcim/netbox/gunicorn.py netbox.wsgi

Restart=on-failure

RestartSec=30

PrivateTmp=true

[Install]

WantedBy=multi-user.target

[Brian Candler]

Google "systemd exit status 203" and you'll see it means it couldn't start the process.  The error is clearly visible:

```

Nov  5 19:43:09 server1 systemd[19442]: netbox.service: Failed to execute command: Permission denied

Nov  5 19:43:09 server1 systemd[19442]: netbox.service: Failed at step EXEC spawning /opt/dcim/netbox/venv/bin/gunicorn: Permission denied

```

It could be that the permissions are wrong, i.e. it doesn't have the execute bit set (which it should do, if it was installed for you by running upgrade.sh)

But most likely it is SELinux which is blocking this - use "dmesg" to look for violations and you'll know if that's the case.

You can put SELinux into permissive mode.  If you don't want to do that, then you'll need to write an SELinux policy for running netbox (good luck with that!!)

[Andrew Meyer]

There is nothing in dmesg.  Also, I tried installing this on AlmaLinux and Vanilla Redhat 8.4.1 images.  No issues.

[Brian Candler]

"No issues" means it works with AlmaLinux and vanilla Redhat 8.4.1 ?

In that case, clearly there's something different about the other RH 8.4.1 system where the problem is. I'm afraid it's not really a Netbox issue then, but an issue with that particular system.

If you "su" to the netbox user, and run /opt/dcim/netbox/venv/bin/gunicorn at the command line, you should see the same issue.

[Andrew Meyer]

Yeah I know.  But I can't pin down the issue.  :-(