Upgrade fail

[Nguyen Quoc Van]

Hi,

I upgrade Netbox from 2.10.6 to latest by using git and upgrade.sh script. The result is successfully but Netbox run error. I had checked and seen netbox-rq service stopped. It's show many error:

 File "/opt/netbox/netbox/circuits/api/urls.py", line 2, in <module>

    from . import views

  File "/opt/netbox/netbox/circuits/api/views.py", line 5, in <module>

    from dcim.api.views import PassThroughPortMixin

  File "/opt/netbox/netbox/dcim/api/views.py", line 30, in <module>

    from . import serializers

  File "/opt/netbox/netbox/dcim/api/serializers.py", line 6, in <module>

    from timezone_field.rest_framework import TimeZoneSerializerField

ModuleNotFoundError: No module named 'timezone_field.rest_framework'

I also try install missing package but it not affected

(venv) nqvan:/opt/netbox$ sudo pip install django-timezone-field

Requirement already satisfied: django-timezone-field in /usr/local/lib/python3.8/dist-packages (4.0)

Requirement already satisfied: django>=2.2 in /usr/local/lib/python3.8/dist-packages (from django-timezone-field) (3.1.3)

Requirement already satisfied: pytz in /usr/local/lib/python3.8/dist-packages (from django-timezone-field) (2021.1)

Requirement already satisfied: sqlparse>=0.2.2 in /usr/local/lib/python3.8/dist-packages (from django>=2.2->django-timezone-field) (0.4.1)

Requirement already satisfied: asgiref<4,>=3.2.10 in /usr/local/lib/python3.8/dist-packages (from django>=2.2->django-timezone-field) (3.3.1)

WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

How can i do that?

[Brian Candler]

When you do "sudo pip" you are dropping out of the virtual environment, so you're looking at globally installed packages (note the paths shown start with /usr/local/lib and not /opt/netbox/venv)

You need to either:

(1)

sudo -s   # switch to root first

cd /opt/netbox

source venv/bin/activate

pip install blah

exit

(2)

# already inside a virtualenv, I *think* this will work

sudo -E pip install blah

(3)

# as root

cd /opt/netbox

echo "django-timezone-field" >>local_requirements.txt

./upgrade.sh

Option 3 is preferred because the extra requirements will persist through future version upgrades.

[Nguyen Quoc Van]

Hi  Brian Candler

Does default upgrade script fail? I upgrade by using git with command below:

cd /opt/netbox 

sudo git checkout master 

sudo git pull origin master

sudo ./upgrade.sh

This script upgrade successfully but netbox-rq service start error

Regards

Vào lúc 15:16:05 UTC+7 ngày Thứ Bảy, 7 tháng 8, 2021, Brian Candler đã viết:

[Nguyen Quoc Van]

Hi

I also try this

(3)

# as root

cd /opt/netbox

echo "django-timezone-field" >>local_requirements.txt

./upgrade.sh

but it is still show error 

File "<frozen importlib._bootstrap>", line 1014, in _gcd_import

  File "<frozen importlib._bootstrap>", line 991, in _find_and_load

  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked

  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked

  File "<frozen importlib._bootstrap_external>", line 848, in exec_module

  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed

  File "/opt/netbox/netbox/netbox/urls.py", line 51, in <module>

    path('api/circuits/', include('circuits.api.urls')),

  File "/usr/local/lib/python3.8/dist-packages/django/urls/conf.py", line 34, in include

    urlconf_module = import_module(urlconf_module)

  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module

    return _bootstrap._gcd_import(name[level:], package, level)

  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import

  File "<frozen importlib._bootstrap>", line 991, in _find_and_load

  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked

  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked

  File "<frozen importlib._bootstrap_external>", line 848, in exec_module

  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed

  File "/opt/netbox/netbox/circuits/api/urls.py", line 2, in <module>

    from . import views

  File "/opt/netbox/netbox/circuits/api/views.py", line 5, in <module>

    from dcim.api.views import PassThroughPortMixin

  File "/opt/netbox/netbox/dcim/api/views.py", line 30, in <module>

    from . import serializers

  File "/opt/netbox/netbox/dcim/api/serializers.py", line 6, in <module>

    from timezone_field.rest_framework import TimeZoneSerializerField

ModuleNotFoundError: No module named 'timezone_field.rest_framework'

(venv) root:/opt/netbox# pip install django-timezone-field

Requirement already satisfied: django-timezone-field in ./venv/lib/python3.8/site-packages (4.1.2)

Requirement already satisfied: pytz in ./venv/lib/python3.8/site-packages (from django-timezone-field) (2021.1)

Requirement already satisfied: django>=2.2 in ./venv/lib/python3.8/site-packages (from django-timezone-field) (3.2.5)

Requirement already satisfied: sqlparse>=0.2.2 in ./venv/lib/python3.8/site-packages (from django>=2.2->django-timezone-field) (0.4.1)

Requirement already satisfied: asgiref<4,>=3.3.2 in ./venv/lib/python3.8/site-packages (from django>=2.2->django-timezone-field) (3.4.1)

Vào lúc 16:50:07 UTC+7 ngày Thứ Bảy, 7 tháng 8, 2021, Nguyen Quoc Van đã viết:

[Brian Candler]

What does "git status" show?

Did "git pull origin master" show any errors?

[Nguyen Quoc Van]

Hi

root:/opt/netbox# git status

On branch master

Your branch is up to date with 'origin/master'.

Changes not staged for commit:

  (use "git add <file>..." to update what will be committed)

  (use "git restore <file>..." to discard changes in working directory)

        modified:   netbox/extras/migrations/0051_migrate_customfields.py

no changes added to commit (use "git add" and/or "git commit -a")

git pull origin master not show any error

Vào lúc 19:04:33 UTC+7 ngày Thứ Bảy, 7 tháng 8, 2021, Brian Candler đã viết:

[Brian Candler]

>       modified:   netbox/extras/migrations/0051_migrate_customfields.py

That is weird.  I'd be interested to see "git diff" output.

I suggest you rename /opt/netbox to something else, make a fresh checkout in /opt/netbox, then copy over configuration.py plus any other extra bits needed (e.g. media, scripts, reports)

[Nguyen Quoc Van]

Hi 

/opt/netbox# git diff

diff --git a/netbox/extras/migrations/0051_migrate_customfields.py b/netbox/extras/migrations/0051_migrate_customfields.py

index 41b2febe..0ea29262 100644

--- a/netbox/extras/migrations/0051_migrate_customfields.py

+++ b/netbox/extras/migrations/0051_migrate_customfields.py

@@ -65,6 +65,9 @@ def migrate_customfieldvalues(apps, schema_editor):

         # Read and update custom field value for each instance

         # TODO: This can be done more efficiently once .update() is supported for JSON fields

         cf_data = model.objects.filter(pk=cfv.obj_id).values('custom_field_data').first()

+        if cf_data is None:

+            print(f'{cfv.field.name} ({cfv.field.type}): {cfv.serialized_value} ({cfv.pk}): references non-existent {model.__name__} {cfv.obj_id}')

+            continue

         try:

             cf_data['custom_field_data'][cfv.field.name] = deserialize_value(cfv.field, cfv.serialized_value)

         except Exception as e:

Vào lúc 22:48:58 UTC+7 ngày Thứ Bảy, 7 tháng 8, 2021, Brian Candler đã viết:

[Nguyen Quoc Van]

Hi,

I upgraded from source and it's ok now. Git method still error

Regards

Vào lúc 08:45:27 UTC+7 ngày Chủ Nhật, 8 tháng 8, 2021, Nguyen Quoc Van đã viết:

[Brian Candler]

Ah right... I believe that was a temporary modification I suggested a while ago, due to some other problem you had with migrations and custom field data.

To undo this change:

git checkout netbox/extras/migrations/0051_migrate_customfields.py

Then do:

git pull

This modified file might have been stopping "git pull" from working - or maybe not, but even so you should keep your source in line with the official source to prevent merge problems.

The git method is much easier for keeping Netbox up to date, so I suggest you go back to it.  If there's a problem with your current git checkout, you can always make a fresh git checkout (instead of using the source tarball/zipball)

[Nguyen Quoc Van]

Hi,

I try:

git checkout master

git checkout netbox/extras/migrations/0051_migrate_customfields.py

git pull

It still error :)

Regards

Vào lúc 15:11:15 UTC+7 ngày Chủ Nhật, 8 tháng 8, 2021, Brian Candler đã viết:

[Brian Candler]

And the response to each of those commands?