Error on SCM repo migration from RB 5.0.5 to 6.0.2

[Florian Miedniak]

Hi,

I'm just trying to upgrade from RB 5.0.5 to 6.0.2, which went quite well so far. Though, having a look at the upgraded installation, I discovered a problem with SCM repository settings:

Repos that had been are properly configured for Subversion in RB 5.0.5, are missing the repo-specific settings after upgrade to RB 6.0.2.

- https://.../admin/db/scmtools/repository/ now shows a new column "Type/Account" which contains for all affected repos "subversion (missing support)"

- Settings (Repo type, path, credentials) are simply gone

- When trying to edit an existing repo, in dropdown "Repository type" there isn't an entry for "Subversion" at all.

- When creating a new repo from scratch, the dropdown has an entry "Subversion", rest of configuration then is straight forward and working well.

It seems to me, that there is some kind of (database?) migration missing.

Is there anything I can do to solve this other than simply recreating all the repos? I fear a bit that doing so, relations to existing review records are broken ...

Best regards,

-Florian 

[David Trowbridge]

Florian,

You probably had subvertpy installed previously. Review Board 6 has removed the subvertpy backend because it caused significant problems. You'll need to install PySVN. We have a guide for doing that at https://www.reviewboard.org/docs/manual/latest/admin/installation/linux/#installing-svn

David

--
Supercharge your Review Board with Power Pack: https://www.reviewboard.org/powerpack/
Want us to host Review Board for you? Check out RBCommons: https://rbcommons.com/
Happy user? Let us know! https://www.reviewboard.org/users/
---
You received this message because you are subscribed to the Google Groups "Review Board Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to reviewboard...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/reviewboard/c2d42fc8-db68-4b4e-b9f6-bbd4f8f9cb94n%40googlegroups.com.

[Florian Miedniak]

Hi David,

I'm using the docker images, both for 5.0.5 and 6.0.2. For the 6.0.2 rb container I installed pysvn via Pip.

Since adding a new subversion repo worked just fine, I doubt it's a general problem of pysvn, right?

Any further ideas? Anything I can do to further track it down?

Btw, it would also be ok to manually recreate all repos, as long as existing review records don't get broken...

-Florian

--

Supercharge your Review Board with Power Pack: https://www.reviewboard.org/powerpack/
Want us to host Review Board for you? Check out RBCommons: https://rbcommons.com/
Happy user? Let us know! https://www.reviewboard.org/users/
---

You received this message because you are subscribed to a topic in the Google Groups "Review Board Community" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/reviewboard/zppupZ36uzE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to reviewboard...@googlegroups.com.

[David Trowbridge]

Florian,

If you're seeing "missing support" in the list, it's because pysvn isn't loaded correctly. You'll still be able to create/edit repositories but they won't actually work until that's fixed. Recreating the repository configurations won't change that, and deleting a repository entry will delete all linked data.

You can't just "pip install pysvn". Someone has created a "pysvn" package on pypi, but it's not the same library. You'll need to use our installation script.

David

You received this message because you are subscribed to the Google Groups "Review Board Community" group.

To unsubscribe from this group and stop receiving emails from it, send an email to reviewboard...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/reviewboard/CADq2vJDDsJXGbLQCXshz1h_e0eMcZZ%2BZaZYMUDB82k%3DKXFhDiQ%40mail.gmail.com.

[Florian Miedniak]

Hi David,

thanks for your quick and detailed answer. I now managed to build and install pysvn with the help of your installer script. Output seems fine so far (see below), but in the end the situation is very similar as already described:

- Type/account column is still reading "subversion (missing support)"

- Repo type is only available in dropdown when creating a new repo

- But I discovered a new behaviour: 

reviewboard_1  | [2024-03-04 15:47:43 +0000] [16] [DEBUG] GET /admin/db/scmtools/repository/6/change/
reviewboard_1  | Error loading SCMTool subversion <reviewboard.scmtools.svn.SVNTool>: 'utf-8' codec can't decode byte 0xcf in position 2: invalid continuation byte
reviewboard_1  | Traceback (most recent call last):
reviewboard_1  |   File "/venv/lib/python3.11/site-packages/reviewboard/scmtools/forms.py", line 1002, in __init__
reviewboard_1  |     self._load_scmtool(scmtool_cls=scmtool,
reviewboard_1  |   File "/venv/lib/python3.11/site-packages/reviewboard/scmtools/forms.py", line 1325, in _load_scmtool
reviewboard_1  |     auth_form.load()
reviewboard_1  |   File "/venv/lib/python3.11/site-packages/reviewboard/scmtools/forms.py", line 196, in load
reviewboard_1  |     for key, value in self.get_initial_data().items():
reviewboard_1  |                       ^^^^^^^^^^^^^^^^^^^^^^^
reviewboard_1  |   File "/venv/lib/python3.11/site-packages/reviewboard/scmtools/forms.py", line 350, in get_initial_data
reviewboard_1  |     return self.get_field_data_from(self.repository,
reviewboard_1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
reviewboard_1  |   File "/venv/lib/python3.11/site-packages/reviewboard/scmtools/forms.py", line 249, in get_field_data_from
reviewboard_1  |     data[key] = getattr(obj, key)
reviewboard_1  |                 ^^^^^^^^^^^^^^^^^
reviewboard_1  |   File "/venv/lib/python3.11/site-packages/reviewboard/scmtools/models.py", line 291, in password
reviewboard_1  |     password = decrypt_password(password)
reviewboard_1  |                ^^^^^^^^^^^^^^^^^^^^^^^^^^
reviewboard_1  |   File "/venv/lib/python3.11/site-packages/reviewboard/scmtools/crypto_utils.py", line 192, in decrypt_password
reviewboard_1  |     .decode('utf-8')
reviewboard_1  |      ^^^^^^^^^^^^^^^
reviewboard_1  | UnicodeDecodeError: 'utf-8' codec can't decode byte 0xcf in position 2: invalid continuation byte

Here are the details of the pysvn installation and initial startup of RB:

+ curl https://pysvn.reviewboard.org | python3
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   614  100   614    0     0   3254      0 --:--:-- --:--:-- --:--:--  3265
Looking up latest PySVN version...
Downloading PySVN 1.9.22...
Building PySVN...
Processing /tmp/tmp33uf_m9l.pysvn-install/pysvn-1.9.22
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Building wheels for collected packages: pysvn
  Building wheel for pysvn (setup.py): started
  Building wheel for pysvn (setup.py): finished with status 'done'
  Created wheel for pysvn: filename=pysvn-1.9.22-cp311-cp311-linux_x86_64.whl size=737322 sha256=8e0e6f5b1f585087a3b9d5cc8ae2cf7ece71c95840d1257564439136c4f57e5e
  Stored in directory: /root/.cache/pip/wheels/ee/30/33/73a37c194727dd5e78a1d04594aa2facc7b1fdd7723bbff663
Successfully built pysvn
Installing collected packages: pysvn
  Attempting uninstall: pysvn
    Found existing installation: pysvn 1.9.22
    Uninstalling pysvn-1.9.22:
      Successfully uninstalled pysvn-1.9.22
Successfully installed pysvn-1.9.22
PySVN is installed.

$ docker-compose up

reviewboard_1  | + rb-site upgrade --copy-media /site
reviewboard_1  | postgresql is up!
reviewboard_1  | ————————————————————————————————————————————————————————————————————————————————
reviewboard_1  |  Upgrading Review Board from 5.0.5 to 6.0.2
reviewboard_1  | ————————————————————————————————————————————————————————————————————————————————
reviewboard_1  |
reviewboard_1  | Rebuilding directory structure ... OK
reviewboard_1  |
reviewboard_1  | Updating database. This may take a while. Please be patient and DO NOT CANCEL!
reviewboard_1  |
reviewboard_1  | Applying database evolution for attachments...
reviewboard_1  |
reviewboard_1  | Resetting in-database caches ... OK
reviewboard_1  |
reviewboard_1  |
reviewboard_1  | ————————————————————————————————————————————————————————————————————————————————
reviewboard_1  |  Upgrade complete!
reviewboard_1  | ————————————————————————————————————————————————————————————————————————————————
reviewboard_1  |
reviewboard_1  | + rb-site manage /site install-extension-media -- --force
reviewboard_1  | DEBUG:root:Logging to /site/logs/reviewboard.log with a minimum level of DEBUG
reviewboard_1  | 2024-03-04 15:43:50,314 - DEBUG -  - root - Logging to /site/logs/reviewboard.log with a minimum level of DEBUG
reviewboard_1  | DEBUG:/venv/lib/python3.11/site-packages/rbpowerpack/scmtools/versionvault.pyc:Using cleartool cleartool
reviewboard_1  | 2024-03-04 15:43:50,533 - DEBUG -  - /venv/lib/python3.11/site-packages/rbpowerpack/scmtools/versionvault.pyc - Using cleartool cleartool
reviewboard_1  | DEBUG:django_evolution.evolve.evolve_app_task:New models for rbpowerpack: []
reviewboard_1  | 2024-03-04 15:43:50,882 - DEBUG -  - django_evolution.evolve.evolve_app_task - New models for rbpowerpack: []
reviewboard_1  | DEBUG:django_evolution.evolve.evolve_app_task:New models for sshdb: []
reviewboard_1  | 2024-03-04 15:43:50,899 - DEBUG -  - django_evolution.evolve.evolve_app_task - New models for sshdb: []
reviewboard_1  | DEBUG:reviewboard.scmtools.svn:Using reviewboard.scmtools.svn.pysvn backend for SVN
reviewboard_1  | 2024-03-04 15:43:50,979 - DEBUG -  - reviewboard.scmtools.svn - Using reviewboard.scmtools.svn.pysvn backend for SVN
reviewboard_1  | DEBUG:reviewboard.scmtools.clearcase:Using cleartool cleartool
reviewboard_1  | 2024-03-04 15:43:51,034 - DEBUG -  - reviewboard.scmtools.clearcase - Using cleartool cleartool
reviewboard_1  | DEBUG:djblets.extensions.manager:Forcing installation of extension media for Review Board Power Pack 5.2.2 (enabled = True)
reviewboard_1  | 2024-03-04 15:43:51,045 - DEBUG -  - djblets.extensions.manager - Forcing installation of extension media for Review Board Power Pack 5.2.2 (enabled = True)
reviewboard_1  | ERROR:djblets.extensions.manager:The Review Board Power Pack extension uses the deprecated "htdocs" directory for static files. This is no longer supported. It must be updated to use a "static" directory instead.
reviewboard_1  | 2024-03-04 15:43:51,045 - ERROR -  - djblets.extensions.manager - The Review Board Power Pack extension uses the deprecated "htdocs" directory for static files. This is no longer supported. It must be updated to use a "static" directory instead.
reviewboard_1  | DEBUG:djblets.extensions.manager:Forcing installation of extension media for Review Bot 3.2 (enabled = True)
reviewboard_1  | 2024-03-04 15:43:51,052 - DEBUG -  - djblets.extensions.manager - Forcing installation of extension media for Review Bot 3.2 (enabled = True)
reviewboard_1  | DEBUG:djblets.extensions.manager:Forcing installation of extension media for Review Board Integrations 3.1.1 (enabled = True)
reviewboard_1  | 2024-03-04 15:43:51,054 - DEBUG -  - djblets.extensions.manager - Forcing installation of extension media for Review Board Integrations 3.1.1 (enabled = True)
reviewboard_1  | DEBUG:djblets.extensions.manager:Forcing installation of extension media for Comment Severity 0.6 (enabled = True)
reviewboard_1  | 2024-03-04 15:43:51,059 - DEBUG -  - djblets.extensions.manager - Forcing installation of extension media for Comment Severity 0.6 (enabled = True)
reviewboard_1  | DEBUG:djblets.extensions.manager:Forcing installation of extension media for Note to Reviewers 1.0.1 (enabled = True)
reviewboard_1  | 2024-03-04 15:43:51,061 - DEBUG -  - djblets.extensions.manager - Forcing installation of extension media for Note to Reviewers 1.0.1 (enabled = True)
reviewboard_1  | + chown -R reviewboard:reviewboard /site/data /site/htdocs/media/ext /site/htdocs/media/uploaded /site/htdocs/static/ext /site/logs /site/tmp
reviewboard_1  | + '[' -f '' ']'
reviewboard_1  | + echo 'Running server...'
reviewboard_1  | + exec gosu reviewboard /serve.sh
reviewboard_1  | Running server...
reviewboard_1  | [2024-03-04 15:43:51 +0000] [15] [DEBUG] Current configuration:
reviewboard_1  |   config: ./gunicorn.conf.py
reviewboard_1  |   wsgi_app: None
reviewboard_1  |   bind: ['0.0.0.0:8080']
reviewboard_1  |   backlog: 2048
reviewboard_1  |   workers: 4
reviewboard_1  |   worker_class: sync
reviewboard_1  |   threads: 20
reviewboard_1  |   worker_connections: 1000
reviewboard_1  |   max_requests: 0
reviewboard_1  |   max_requests_jitter: 0
reviewboard_1  |   timeout: 120
reviewboard_1  |   graceful_timeout: 30
reviewboard_1  |   keepalive: 2
reviewboard_1  |   limit_request_line: 4094
reviewboard_1  |   limit_request_fields: 100
reviewboard_1  |   limit_request_field_size: 8190
reviewboard_1  |   reload: False
reviewboard_1  |   reload_engine: auto
reviewboard_1  |   reload_extra_files: []
reviewboard_1  |   spew: False
reviewboard_1  |   check_config: False
reviewboard_1  |   print_config: False
reviewboard_1  |   preload_app: False
reviewboard_1  |   sendfile: None
reviewboard_1  |   reuse_port: False
reviewboard_1  |   chdir: /
reviewboard_1  |   daemon: False
reviewboard_1  |   raw_env: []
reviewboard_1  |   pidfile: None
reviewboard_1  |   worker_tmp_dir: /dev/shm
reviewboard_1  |   user: 1001
reviewboard_1  |   group: 1001
reviewboard_1  |   umask: 0
reviewboard_1  |   initgroups: False
reviewboard_1  |   tmp_upload_dir: None
reviewboard_1  |   secure_scheme_headers: {'X-FORWARDED-PROTOCOL': 'ssl', 'X-FORWARDED-PROTO': 'https', 'X-FORWARDED-SSL': 'on'}
reviewboard_1  |   forwarded_allow_ips: ['127.0.0.1']
reviewboard_1  |   accesslog: None
reviewboard_1  |   disable_redirect_access_to_syslog: False
reviewboard_1  |   access_log_format: %(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s"
reviewboard_1  |   errorlog: -
reviewboard_1  |   loglevel: debug
reviewboard_1  |   capture_output: False
reviewboard_1  |   logger_class: gunicorn.glogging.Logger
reviewboard_1  |   logconfig: None
reviewboard_1  |   logconfig_dict: {}
reviewboard_1  |   logconfig_json: None
reviewboard_1  |   syslog_addr: udp://localhost:514
reviewboard_1  |   syslog: False
reviewboard_1  |   syslog_prefix: None
reviewboard_1  |   syslog_facility: user
reviewboard_1  |   enable_stdio_inheritance: False
reviewboard_1  |   statsd_host: None
reviewboard_1  |   dogstatsd_tags:
reviewboard_1  |   statsd_prefix:
reviewboard_1  |   proc_name: None
reviewboard_1  |   default_proc_name: reviewboard.wsgi
reviewboard_1  |   pythonpath: None
reviewboard_1  |   paste: None
reviewboard_1  |   on_starting: <function OnStarting.on_starting at 0x7f320f010180>
reviewboard_1  |   on_reload: <function OnReload.on_reload at 0x7f320f0102c0>
reviewboard_1  |   when_ready: <function WhenReady.when_ready at 0x7f320f010400>
reviewboard_1  |   pre_fork: <function Prefork.pre_fork at 0x7f320f010540>
reviewboard_1  |   post_fork: <function Postfork.post_fork at 0x7f320f010680>
reviewboard_1  |   post_worker_init: <function PostWorkerInit.post_worker_init at 0x7f320f0107c0>
reviewboard_1  |   worker_int: <function WorkerInt.worker_int at 0x7f320f010900>
reviewboard_1  |   worker_abort: <function WorkerAbort.worker_abort at 0x7f320f010a40>
reviewboard_1  |   pre_exec: <function PreExec.pre_exec at 0x7f320f010b80>
reviewboard_1  |   pre_request: <function PreRequest.pre_request at 0x7f320f010cc0>
reviewboard_1  |   post_request: <function PostRequest.post_request at 0x7f320f010d60>
reviewboard_1  |   child_exit: <function ChildExit.child_exit at 0x7f320f010ea0>
reviewboard_1  |   worker_exit: <function WorkerExit.worker_exit at 0x7f320f010fe0>
reviewboard_1  |   nworkers_changed: <function NumWorkersChanged.nworkers_changed at 0x7f320f011120>
reviewboard_1  |   on_exit: <function OnExit.on_exit at 0x7f320f011260>
reviewboard_1  |   ssl_context: <function NewSSLContext.ssl_context at 0x7f320f011440>
reviewboard_1  |   proxy_protocol: False
reviewboard_1  |   proxy_allow_ips: ['127.0.0.1']
reviewboard_1  |   keyfile: None
reviewboard_1  |   certfile: None
reviewboard_1  |   ssl_version: 2
reviewboard_1  |   cert_reqs: 0
reviewboard_1  |   ca_certs: None
reviewboard_1  |   suppress_ragged_eofs: True
reviewboard_1  |   do_handshake_on_connect: False
reviewboard_1  |   ciphers: None
reviewboard_1  |   raw_paste_global_conf: []
reviewboard_1  |   strip_header_spaces: False
reviewboard_1  | [2024-03-04 15:43:51 +0000] [15] [INFO] Starting gunicorn 21.2.0
reviewboard_1  | [2024-03-04 15:43:51 +0000] [15] [DEBUG] Arbiter booted
reviewboard_1  | [2024-03-04 15:43:51 +0000] [15] [INFO] Listening at: http://0.0.0.0:8080 (15)
reviewboard_1  | [2024-03-04 15:43:51 +0000] [15] [INFO] Using worker: gthread
reviewboard_1  | [2024-03-04 15:43:51 +0000] [16] [INFO] Booting worker with pid: 16
reviewboard_1  | [2024-03-04 15:43:51 +0000] [17] [INFO] Booting worker with pid: 17
reviewboard_1  | [2024-03-04 15:43:51 +0000] [18] [INFO] Booting worker with pid: 18
reviewboard_1  | [2024-03-04 15:43:51 +0000] [19] [INFO] Booting worker with pid: 19
reviewboard_1  | [2024-03-04 15:43:51 +0000] [15] [DEBUG] 4 workers
reviewboard_1  | [2024-03-04 15:44:18 +0000] [16] [DEBUG] GET /health/
reviewboard_1  | 2024-03-04 15:44:18,085 - DEBUG -  - root - Logging to /site/logs/reviewboard.log with a minimum level of DEBUG
reviewboard_1  | 2024-03-04 15:44:18,290 - DEBUG -  - /venv/lib/python3.11/site-packages/rbpowerpack/scmtools/versionvault.pyc - Using cleartool cleartool
reviewboard_1  | 2024-03-04 15:44:18,426 - DEBUG -  - reviewboard.scmtools.svn - Using reviewboard.scmtools.svn.pysvn backend for SVN
reviewboard_1  | 2024-03-04 15:44:18,435 - DEBUG -  - reviewboard.scmtools.clearcase - Using cleartool cleartool
reviewboard_1  | 2024-03-04 15:44:18,446 - DEBUG -  - root - Reloading logging settings

Do you have any idea what am I still missing / what is still broken in my installation ?

-Florian

[David Trowbridge]

Was this migrated from another server? If so, can you check that the SECRET_KEY is the same in the reviewboard config file?

David

To view this discussion on the web visit https://groups.google.com/d/msgid/reviewboard/607ecfa5-db86-4d50-ab2e-56ad5cd3a3adn%40googlegroups.com.

[Florian Miedniak]

Yes, it was migrated from production to a staging environment and there was some misconfiguration of docker volumes causing ReviewBoard to re-create the secret.

After fixing the docker volume configuration and filling with the data and configuration file from production it finally works like a charm! :-)

Thank you for your kind and very responsive support!

Just a further question as I'm curious (and a bit confused):

I head a look at https://github.com/reviewboard/reviewboard/blob/release-6.0.2/contrib/docker/Dockerfile and found that pysvn building and installation already is there. Does that mean that the official docker image is being built using another Dockerfile?

-Florian

[David Trowbridge]

Glad it's all working! Please let us know if there's anything else we can help with.

David

To view this discussion on the web visit https://groups.google.com/d/msgid/reviewboard/d58729ff-728b-4d3c-a8f0-6e05b213b46dn%40googlegroups.com.