NetBox v2.7.9 Released
259 views
Skip to first unread message
Jeremy Stretch
Mar 6, 2020, 11:28:08 AM3/6/20
to NetBox
Note: This release will deploy a Python virtual environment on upgrade in the venv/
directory. This will require modifying the paths to your Python and
gunicorn executables in the systemd service files. For more detail,
please see the upgrade instructions.
Enhancements
- #3949 - Revised the installation docs and upgrade script to employ a Python virtual environment
- #4062 - Enumerate ChoiceField type and value in API
- #4119 - Extend upgrade script to clear expired user sessions
- #4121 - Add dynamic lookup expressions for all filters
- #4218 - Allow negative voltage for DC power feeds
- #4281 - Allow filtering device component list views by type
- #4284 - Add MRJ21 port and cable types
- #4290 - Include device name in tooltip on rack elevations
- #4305 - Add 10-inch option for rack width
Bug Fixes
- #4274 - Fix incorrect schema definition of
inttype choicefields - #4277 - Fix filtering of clusters by tenant
- #4282 - Fix label on export button for device types
- #4285 - Include A/Z termination sites in provider circuits table
- #4295 - Fix assignment of parent LAG during interface bulk edit
- #4298 - Fix bulk creation of objects with custom fields via REST API
- #4300 - Pass "commit" argument when executing scripts via REST API
- #4301 - Fix exception when deleting device type with components
- #4306 - Fix toggling of device images for all racks in elevations view
--
Jeremy Stretch
Sr. Network Automation Engineer
Network to Code, LLC
Sr. Network Automation Engineer
Network to Code, LLC
Brian Candler
Mar 6, 2020, 11:43:59 AM3/6/20
to NetBox
I'm not sure this such a breaking change was wise at a point release.
First attempt told me I had to install python3-venv. After fixing this, the next pass broke:
...
Building wheels for collected packages: funcy, inflection, coreschema, itypes, pyrsistent
Running setup.py bdist_wheel for funcy ... error
Complete output from command /opt/netbox/venv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-ovgjkshf/funcy/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" bdist_wheel -d /tmp/tmp0iian73rpip-wheel- --python-tag cp35:
usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: -c --help [cmd1 cmd2 ...]
or: -c --help-commands
or: -c cmd --help
error: invalid command 'bdist_wheel'
----------------------------------------
Failed building wheel for funcy
Running setup.py clean for funcy
Running setup.py bdist_wheel for inflection ... error
Complete output from command /opt/netbox/venv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-ovgjkshf/inflection/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" bdist_wheel -d /tmp/tmpjamrkmy7pip-wheel- --python-tag cp35:
usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: -c --help [cmd1 cmd2 ...]
or: -c --help-commands
or: -c cmd --help
error: invalid command 'bdist_wheel'
----------------------------------------
Failed building wheel for inflection
... lots more similar errors
"python3-wheel" was already installed. Installing "python-wheel-common" didn't make any difference either. And I see "wheel==0.34.2" is already in requirements.txt
I moved "wheel==0.34.2" to be the first line in requirements.txt, and I can see it now installs it first, but that doesn't fix the problem either:
root@netbox:/opt/netbox# ./upgrade.sh
Removing old virtual environment...
Creating a new virtual environment at /opt/netbox/venv...
Installing Python packages (pip3 install -r requirements.txt)...
Collecting wheel==0.34.2 (from -r requirements.txt (line 1))
Collecting Django<2.3,>=2.2 (from -r requirements.txt (line 2))
...
Collecting zipp>=0.5 (from importlib-metadata; python_version < "3.8"->jsonschema->swagger-spec-validator>=2.1.0; extra == "validation"->drf-yasg[validation]==1.17.0->-r requirements.txt (line 16))
Building wheels for collected packages: funcy, inflection, coreschema, itypes, pyrsistent
Running setup.py bdist_wheel for funcy ... error
Complete output from command /opt/netbox/venv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-4sr69db8/funcy/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" bdist_wheel -d /tmp/tmpmdnjvjdgpip-wheel- --python-tag cp35:
usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: -c --help [cmd1 cmd2 ...]
or: -c --help-commands
or: -c cmd --help
error: invalid command 'bdist_wheel'
----------------------------------------
Failed building wheel for funcy
Running setup.py clean for funcy
Running setup.py bdist_wheel for inflection ... error
Complete output from command /opt/netbox/venv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-4sr69db8/inflection/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" bdist_wheel -d /tmp/tmp072st46xpip-wheel- --python-tag cp35:
usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: -c --help [cmd1 cmd2 ...]
or: -c --help-commands
or: -c cmd --help
error: invalid command 'bdist_wheel'
... etc
Any ideas what to try next? This is under Ubuntu 16.04 BTW. Fortunately this is only my home instance so the outage isn't a problem.
Jeremy Stretch
Mar 6, 2020, 11:54:06 AM3/6/20
to Brian Candler, NetBox
> I'm not sure this such a breaking change was wise at a point release.
It's not intended to be a breaking change. I've tested the installation and upgrade processes extensively and have not run into any issues with the current setup. Unfortunately we no longer package beta releases as community participation has been extremely low.
> Any ideas what to try next?
At what point is the wheel packaging getting installed in the virtual environment? Before for after the errors? Also, what version of pip are you running? I haven't encountered any issues with it but maybe we need to force the installation of wheel first separately.
--
You received this message because you are subscribed to the Google Groups "NetBox" group.
To unsubscribe from this group and stop receiving emails from it, send an email to netbox-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/netbox-discuss/c9c97710-a119-414c-b065-0db8f90c58fe%40googlegroups.com.
Brian Candler
Mar 6, 2020, 12:27:47 PM3/6/20
to NetBox
On Friday, 6 March 2020 16:54:06 UTC, Jeremy Stretch wrote:
At what point is the wheel packaging getting installed in the virtual environment? Before for after the errors?
I moved "wheel" to the top of requirements.txt so it gets installed first, but the same errors occur later on.
Also, what version of pip are you running? I haven't encountered any issues with it but maybe we need to force the installation of wheel first separately.
ii python3-pip 8.1.1-2ubuntu0.4 all alternative Python package installer - Python 3 version of the package
I had previously done "pip3 install pip" to upgrade it:
pip (20.0.2) - The PyPA recommended tool for installing Python packages.
INSTALLED: 19.0.2
LATEST: 20.0.2
but of course that would be ignored inside the venv.
Now I've added "pip>=19.0.0" to the top of reqiurements.txt, but that hasn't stopped the errors either.
Full transcript here:
Brian Candler
Mar 6, 2020, 12:34:54 PM3/6/20
to NetBox
Solved.
I reverted requirements.txt and was able to make it work by upgrading pip *after* the virtualenv is built but *before* everything else is installed:
diff --git a/upgrade.sh b/upgrade.sh
index bd1a06f..3aa1610 100755
--- a/upgrade.sh
+++ b/upgrade.sh
@@ -30,6 +30,7 @@ eval $COMMAND || {
source "${VIRTUALENV}/bin/activate"
# Install Python packages
+pip3 install --upgrade pip
COMMAND="pip3 install -r requirements.txt"
echo "Installing Python packages ($COMMAND)..."
eval $COMMAND || exit 1
Results:
...
Collecting zipp>=0.5
Downloading zipp-1.2.0-py2.py3-none-any.whl (4.8 kB)
Installing collected packages: sqlparse, pytz, Django, six, redis, funcy, django-cacheops, django-cors-headers, django-debug-toolbar, django-filter, django-js-asset, django-mptt, django-pglocks, prometheus-client, django-prometheus, click, rq, django-rq, django-tables2, django-taggit, django-taggit-serializer, django-timezone-field, djangorestframework, pyparsing, packaging, itypes, urllib3, certifi, idna, chardet, requests, uritemplate, MarkupSafe, Jinja2, coreschema, coreapi, ruamel.yaml.clib, ruamel.yaml, inflection, pyrsistent, zipp, importlib-metadata, attrs, jsonschema, PyYAML, swagger-spec-validator, drf-yasg, gunicorn, Markdown, netaddr, Pillow, psycopg2-binary, py-gfm, pycryptodome, svgwrite, wheel
Running setup.py install for funcy ... done
Running setup.py install for itypes ... done
Running setup.py install for coreschema ... done
Running setup.py install for inflection ... done
Running setup.py install for pyrsistent ... done
Successfully installed Django-2.2.11 Jinja2-2.10.3 Markdown-2.6.11 MarkupSafe-1.1.1 Pillow-7.0.0 PyYAML-5.3 attrs-19.3.0 certifi-2019.11.28 chardet-3.0.4 click-7.0 coreapi-2.3.3 coreschema-0.0.4 django-cacheops-4.2 django-cors-headers-3.2.1 django-debug-toolbar-2.1 django-filter-2.2.0 django-js-asset-1.2.2 django-mptt-0.9.1 django-pglocks-1.0.4 django-prometheus-1.1.0 django-rq-2.2.0 django-tables2-2.2.1 django-taggit-1.2.0 django-taggit-serializer-0.1.7 django-timezone-field-4.0 djangorestframework-3.10.3 drf-yasg-1.17.0 funcy-1.14 gunicorn-20.0.4 idna-2.9 importlib-metadata-1.5.0 inflection-0.3.1 itypes-1.1.0 jsonschema-3.2.0 netaddr-0.7.19 packaging-20.3 prometheus-client-0.7.1 psycopg2-binary-2.8.4 py-gfm-0.1.4 pycryptodome-3.9.4 pyparsing-2.4.6 pyrsistent-0.15.7 pytz-2019.3 redis-3.3.11 requests-2.23.0 rq-1.2.2 ruamel.yaml-0.16.10 ruamel.yaml.clib-0.2.0 six-1.14.0 sqlparse-0.3.1 svgwrite-1.3.1 swagger-spec-validator-2.5.0 uritemplate-3.0.1 urllib3-1.25.8 wheel-0.34.2 zipp-1.2.0
Applying database migrations (python3 netbox/manage.py migrate)...
/opt/netbox/netbox/netbox/settings.py:31: UserWarning: Python 3.6 or higher will be required starting with NetBox v2.8 (current: Python 3.5.2)
platform.python_version()
...
Jeremy Stretch
Mar 6, 2020, 12:39:08 PM3/6/20
to Brian Candler, NetBox
Looking at the transcript you provided, it seems that although some of the wheel-based installations are failing, those packages are failing back to using setup.py and still ultimately getting installed successfully even with the older release of pip. I've checked this by comparing line 188 ("Successfully installed...") against requirements.txt and everything is there (unless I've missed something). Can you confirm?
But yes, to avoid the errors entirely we should extend the upgrade script to include updating pip.
--
You received this message because you are subscribed to the Google Groups "NetBox" group.
To unsubscribe from this group and stop receiving emails from it, send an email to netbox-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/netbox-discuss/a62e0ab6-326b-4cc7-b885-343c34da3c02%40googlegroups.com.
Brian Candler
Mar 6, 2020, 1:09:37 PM3/6/20
to NetBox
On Friday, 6 March 2020 17:39:08 UTC, Jeremy Stretch wrote:
Looking at the transcript you provided, it seems that although some of the wheel-based installations are failing, those packages are failing back to using setup.py and still ultimately getting installed successfully even with the older release of pip. I've checked this by comparing line 188 ("Successfully installed...") against requirements.txt and everything is there (unless I've missed something). Can you confirm?
I've reverted requirements.txt, re-run upgrade.sh, ignored the big red errors.
As you say, I see after the failed installs, setup.py is run for each:
Running setup.py install for funcy ... done
Running setup.py install for inflection ... done
Running setup.py install for coreschema ... done
Running setup.py install for itypes ... done
Running setup.py install for pyrsistent ... done
And yes, Netbox does run, so it's OK.
As it happens I saw the same "Running setup.py..." lines with the upgraded pip - although it seems to generate a lot less debug output in general. It could just be that the upgraded pip is hiding the errors.
Incidentally, there's a bug which means that the missing venv warning is printed, even when the venv is present. In upgrade.sh:
if [ WARN_MISSING_VENV ]; then
I think it should be something like:
if [ -n "$WARN_MISSING_VENV" ]; then
Cheers,
Brian.
P.S. I do consider something which makes a fundamental change to the way the application is deployed, and which requires the user to make changes to init scripts, to be a "breaking change".
Brian Candler
Mar 6, 2020, 1:27:47 PM3/6/20
to NetBox
I think I found a better solution: instead of upgrading pip, install wheel *before* installing everything else from requirements.txt
# Install Python packages
+pip3 install wheel
COMMAND="pip3 install -r requirements.txt"
Given this, the results are:
...
Building wheels for collected packages: funcy, inflection, coreschema, itypes, pyrsistent
Running setup.py bdist_wheel for funcy ... done
Stored in directory: /root/.cache/pip/wheels/20/5a/d8/1d875df03deae6f178dfdf70238cca33f948ef8a6f5209f2eb
Running setup.py bdist_wheel for inflection ... done
Stored in directory: /root/.cache/pip/wheels/9f/5a/d3/6fc3bf6516d2a3eb7e18f9f28b472110b59325f3f258fe9211
Running setup.py bdist_wheel for coreschema ... done
Stored in directory: /root/.cache/pip/wheels/10/7b/ba/04fcd6b33e6123ca11a5f5ab56decb1a2d87ced028377a1377
Running setup.py bdist_wheel for itypes ... done
Stored in directory: /root/.cache/pip/wheels/7b/52/af/4e27324812e7ab7bbbc30f748d317f3739477562325cb4c723
Running setup.py bdist_wheel for pyrsistent ... done
Stored in directory: /root/.cache/pip/wheels/b5/78/ac/f26a78a989cd97f90981d96a560d7e1da5e1307284301d94e8
Successfully built funcy inflection coreschema itypes pyrsistent
...
Of course you can then remove wheel from requirements.txt. And upgrading pip is probably a good idea anyway - but it doesn't appear to be required.
Keenan Tims
Mar 6, 2020, 2:16:45 PM3/6/20
to Brian Candler, NetBox
Since the deployment script destroys the venv when it runs, it also doesn't seem possible to install additional required modules (in my scenario, `django-storages`) into the virtualenv, which causes migrations to fail and completely breaks deployment. Manually modifying requirements.txt to include the additional required modules does permit installation, but isn't really a good model.
I agree that Netbox should be run in a virtualenv (and had already deployed it this way), but IMHO this should be part of the installation instructions and not something managed by Netbox's own scripts. It's a deployment detail, not part of the application.
+1 on this being too significant a change for a point release.
Keenan
I agree that Netbox should be run in a virtualenv (and had already deployed it this way), but IMHO this should be part of the installation instructions and not something managed by Netbox's own scripts. It's a deployment detail, not part of the application.
+1 on this being too significant a change for a point release.
Keenan
--
You received this message because you are subscribed to the Google Groups "NetBox" group.
To unsubscribe from this group and stop receiving emails from it, send an email to netbox-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/netbox-discuss/3dd820e1-a9a1-4630-a966-8c5bbf89beaa%40googlegroups.com.
Jeremy Stretch
Mar 6, 2020, 2:48:26 PM3/6/20
to Keenan Tims, Brian Candler, NetBox
> Since the deployment script destroys the venv when it runs, it also doesn't seem possible to install additional required modules
That's a fair point, and we should come up with a good answer for it. Whether that's to persist the virtual environment or to provide a file enumerating local dependencies, I don't know. I'm sure there will be strong opinions either way.
> It's a deployment detail, not part of the application.
It used to be a deployment detail, and is now part of the prescribed installation process. This change was prompted by the recent rise in installation problems that have been raised pertaining to pip dependency conflicts. While a user may continue to use NetBox outside a virtual environment if they choose, it precludes them from using the packaged upgrade script and relying on a portion of the official documentation.
> +1 on this being too significant a change for a point release.
NetBox versioning pertains only to the data model and API. It does not imply anything about the upgrade procedure for a release. Routine administrative tasks such as this may be bundled with point releases, and will be highlighted appropriately in the release notes as this one was.
To view this discussion on the web visit https://groups.google.com/d/msgid/netbox-discuss/a854bca5d51dc1ea8e8818eda1686460%40gotroot.ca.
Reply all
Reply to author
Forward
0 new messages