Migrating to a new server

[Alfred von Campe]

I must be missing a simple step somewhere. I’m trying to migrate a ReviewBoard version 1.7.27 instance running on CentOS 6 to version 2.5.6.1 on CentOS 7. I’ve installed ReviewBoard (via EPEL RPM) on the new server, installed MySQL and created the reviewboard database, loaded the SQL dump from the old server, copied the site directory to the new server, and am trying to run “rb-site upgrade”. This fails with the following error: "django.db.utils.OperationalError: (1045, "Access denied for user 'rb'@'localhost' (using password: YES)”)”, which make sense because I haven’t configured the password anywhere. Where does it go?

The documentation is great about how to set up a new site, but I can’t find anything about migrating from one server to another. Is there such a document?

Alfred

[Christian Hammond]

Hi Alfred,

There's a draft of a document on my computer, but nothing online yet. I need to finish it.

The database password is stored in $sitedir/conf/settings_local.py. Sounds like the database was copied but the "rb" database user wasn't recreated. Once that's fixed, you should be able to perform the upgrade.

Christian

--
Supercharge your Review Board with Power Pack: https://www.reviewboard.org/powerpack/
Want us to host Review Board for you? Check out RBCommons: https://rbcommons.com/
Happy user? Let us know! https://www.reviewboard.org/users/
---
You received this message because you are subscribed to the Google Groups "reviewboard" group.
To unsubscribe from this group and stop receiving emails from it, send an email to reviewboard...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

-- 
Christian Hammond

President/CEO of Beanbag

Makers of Review Board

[Alfred von Campe]

> On Aug 11, 2016, at 19:41, Christian Hammond <chri...@beanbaginc.com> wrote:
>
> There's a draft of a document on my computer, but nothing online yet. I need to finish it.

If you want me to review/edit it as I’m doing the migration, I will be glad to. Just let me know.

> The database password is stored in $sitedir/conf/settings_local.py. Sounds like the database was copied but the "rb" database user wasn't recreated. Once that's fixed, you should be able to perform the upgrade.

Ah, that’s where it is. I should have known. In any event, I thought I used the same password but I was off by one character. In order to make the migration easier (I plan to do it multiple times until I get it right), I updated the password for the rb user on the new server to match the old one, so I can just copy the site directory as is from the older server to the new one. However, after I did that, I got the errors below running the rb-site upgrade command. I must have missed one minor step when creating the database, but don’t know what.

Thanks,
Alfred



# rb-site upgrade /var/www/html/rb
Rebuilding directory structure
Updating database. This may take a while.

The log output below, including warnings and errors,
can be ignored unless upgrade fails.

------------------ <begin log output> ------------------
Creating tables ...
Creating table accounts_trophy

[!] There was an error synchronizing the database. Make sure the
database is created and has the appropriate permissions, and then
continue.
[!] Details: (1005, "Can't create table 'reviewboard.#sql-41fd_f'
(errno: 150)")

Press Enter to continue
Creating tables ...
Creating table attachments_fileattachmenthistory
Creating table diffviewer_rawfilediffdata
Creating table notifications_webhooktarget_repositories

[!] There was an error synchronizing the database. Make sure the
database is created and has the appropriate permissions, and then
continue.
[!] Details: (1005, "Can't create table 'reviewboard.#sql-41fd_f'
(errno: 150)")

Press Enter to continue
Creating tables ...
Creating table notifications_webhooktarget

[!] There was an error synchronizing the database. Make sure the
database is created and has the appropriate permissions, and then
continue.
[!] Details: (1005, "Can't create table 'reviewboard.#sql-41fd_f'
(errno: 150)")

Press Enter to continue
Creating tables ...
Creating table webapi_webapitoken

[!] There was an error synchronizing the database. Make sure the
database is created and has the appropriate permissions, and then
continue.
[!] Details: (1005, "Can't create table 'reviewboard.#sql-41fd_f'
(errno: 150)")

Press Enter to continue
Creating tables ...
Upgrading Review Board from 1.7.27 to 2.5.6.1
There are unapplied evolutions for auth.
There are unapplied evolutions for contenttypes.
There are unapplied evolutions for accounts.
There are unapplied evolutions for attachments.
There are unapplied evolutions for changedescs.
There are unapplied evolutions for diffviewer.
There are unapplied evolutions for hostingsvcs.
There are unapplied evolutions for notifications.
There are unapplied evolutions for reviews.
There are unapplied evolutions for scmtools.
There are unapplied evolutions for webapi.
Adding baseline version for new models
Evolutions in webapi baseline: webapitoken_jsonfield_null
Evolutions in notifications baseline: webhooktarget_extra_state, webhooktarget_extra_data_null
Project signature has changed - an evolution is required
Installing custom SQL ...
Installing indexes ...
Installed 0 object(s) from 0 fixture(s)
CommandError: Error applying evolution: (1005, "Can't create table 'reviewboard.#sql-41fd_10' (errno: 150)")

[Christian Hammond]

Hi Alfred,

Sorry for the delay.

This looks like a MyISAM vs. InnoDB table format issue. You'll need to either make sure all your tables are converted to InnoDB (you can replace "MyISAM" with "InnoDB" in the database dump when you go to load it back in), or configure MySQL to use MyISAM by default (though we recommend InnoDB).

You'll need to fix things up and retry the upgrade from the backup.

Christian

-- 
Christian Hammond

President/CEO of Beanbag

Makers of Review Board

--
Supercharge your Review Board with Power Pack: https://www.reviewboard.org/powerpack/
Want us to host Review Board for you? Check out RBCommons: https://rbcommons.com/
Happy user? Let us know! https://www.reviewboard.org/users/
---
You received this message because you are subscribed to the Google Groups "reviewboard" group.

To unsubscribe from this group and stop receiving emails from it, send an email to reviewboard+unsubscribe@googlegroups.com.

[Alfred von Campe]

On Aug 14, 2016, at 19:55, Christian Hammond <chri...@beanbaginc.com> wrote:

This looks like a MyISAM vs. InnoDB table format issue. You'll need to either make sure all your tables are converted to InnoDB (you can replace "MyISAM" with "InnoDB" in the database dump when you go to load it back in), or configure MySQL to use MyISAM by default (though we recommend InnoDB).

You'll need to fix things up and retry the upgrade from the backup.

Yup, that was it.  I took another database dump, used sed to change all instances of “MyISAM” to “InnoDB”, and re-imported the database dump.  This time the “rb-site upgrade” completed successfully (see log output at the end of this message). The web server appears to be configured properly as I can access the site and look at all the existing reviews.  However, when I try to log into the site, I get the following error:

Something broke! (Error 500)

It appears something broke when you tried to go to here. This is either a bug in Review Board or a server configuration error. Please report this to your administrator.

We are authenticating against our corporate AD servers, and this works fine on the existing (CentOS 6 and RB V1.7.27) ReviewBoard instance.  It’s only on the new instance (CentOS 7 and RB V2.5.6.1) that I get this issue.  I must have missed a configuration setting somewhere.  Any ideas what that might be or what log files I should be examining to figure out where the problem is?

Alfred

Output from “rb-site upgrade”:   

Rebuilding directory structure

Updating database. This may take a while.

The log output below, including warnings and errors,

can be ignored unless upgrade fails.

------------------ <begin log output> ------------------

Creating tables ...

Creating table accounts_trophy

Creating table attachments_fileattachmenthistory

Creating table diffviewer_rawfilediffdata

Creating table notifications_webhooktarget_repositories

Creating table notifications_webhooktarget

Creating table webapi_webapitoken

Upgrading Review Board from 1.7.27 to 2.5.6.1

There are unapplied evolutions for auth.

There are unapplied evolutions for contenttypes.

There are unapplied evolutions for accounts.

There are unapplied evolutions for attachments.

There are unapplied evolutions for changedescs.

There are unapplied evolutions for diffviewer.

There are unapplied evolutions for hostingsvcs.

There are unapplied evolutions for notifications.

There are unapplied evolutions for reviews.

There are unapplied evolutions for scmtools.

There are unapplied evolutions for webapi.

Adding baseline version for new models

Evolutions in webapi baseline: webapitoken_jsonfield_null

Evolutions in notifications baseline: webhooktarget_extra_state, webhooktarget_extra_data_null

Project signature has changed - an evolution is required

Installing custom SQL ...

Installing indexes ...

Installed 0 object(s) from 0 fixture(s)

Evolution successful.

------------------- <end log output> -------------------

[Christian Hammond]

Hi Alfred,

The Review Board log files ($sitedir/logs) should include some information. Failing that, the Apache error log.

You may have to install the appropriate Python modules for Active Directory again. (I wouldn't expect a crash, but there may be a regression.)

Christian

-- 
Christian Hammond

President/CEO of Beanbag

Makers of Review Board

--

[Alfred von Campe]

On Aug 15, 2016, at 16:21, Christian Hammond <chri...@beanbaginc.com> wrote:

The Review Board log files ($sitedir/logs) should include some information. Failing that, the Apache error log.

You may have to install the appropriate Python modules for Active Directory again. (I wouldn't expect a crash, but there may be a regression.)

As usual, your advice was right on the money.  I checked the reviewboard.log file and there was a "ImportError: No module named ldap” error.  That was easily remedied on the CentOS 7 system with a "yum install python-ldap”.  However, the login is still failing, this time with a "NameError: global name 'filter_format' is not defined” error in the log file.  Any ideas what could be causing that?

Alfred

[Christian Hammond]

What version of ldap do you have? It sounds like it may be too old. Sometimes the distro-provided packages are too stale to rely upon.

I'd recommend uninstalling the RPM and instead installing using pip:

    yum remove python-ldap

    pip install python-ldap

(You may want to upgrade pip before doing this: pip install -U pip).

Christian

-- 
Christian Hammond

President/CEO of Beanbag

Makers of Review Board

--

[Alfred von Campe]

On Aug 15, 2016, at 17:31, Christian Hammond <chri...@beanbaginc.com> wrote:

What version of ldap do you have? It sounds like it may be too old. Sometimes the distro-provided packages are too stale to rely upon.

I'd recommend uninstalling the RPM and instead installing using pip:

Good idea.  The repo based Python LDAP module was version 2.4.15.  Using easy_install (not pip), I got version 2.4.27.  However, I’m now back to the "ImportError: No module named ldap” error, even though the easy_Install appears to have been successful;

# easy_install  python-ldap

.

.

.

Adding python-ldap 2.4.27 to easy-install.pth file

Installed /usr/lib/python2.7/site-packages/python_ldap-2.4.27-py2.7-linux-x86_64.egg

Processing dependencies for python-ldap

Finished processing dependencies for python-ldap

I then installed pip via easy_install, and when I tried to do a “pip install python-ldap” it reports that the requirement is already satisfied:

# pip install python-ldap

Requirement already satisfied (use --upgrade to upgrade): python-ldap in /usr/lib/python2.7/site-packages/python_ldap-2.4.27-py2.7-linux-x86_64.egg

Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/site-packages (from python-ldap)

So it appears that the ldap module is installed (and a “import ldap” from Python confirms it), but ReviewBoard can’t seem to find it:

2016-08-15 21:53:22,998 - ERROR -  - Exception thrown for user AnonymousUser at http://reviewboard.bose.com/account/login/

No module named ldap

Traceback (most recent call last):

  File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 112, in get_response

    response = wrapped_callback(request, *callback_args, **callback_kwargs)

  File "/usr/lib/python2.7/site-packages/django/views/decorators/debug.py", line 75, in sensitive_post_parameters_wrapper

    return view(request, *args, **kwargs)

  File "/usr/lib/python2.7/site-packages/django/utils/decorators.py", line 99, in _wrapped_view

    response = view_func(request, *args, **kwargs)

  File "/usr/lib/python2.7/site-packages/django/views/decorators/cache.py", line 52, in _wrapped_view_func

    response = view_func(request, *args, **kwargs)

  File "/usr/lib/python2.7/site-packages/django/contrib/auth/views.py", line 36, in login

    if form.is_valid():

  File "/usr/lib/python2.7/site-packages/django/forms/forms.py", line 129, in is_valid

    return self.is_bound and not bool(self.errors)

  File "/usr/lib/python2.7/site-packages/django/forms/forms.py", line 121, in errors

    self.full_clean()

  File "/usr/lib/python2.7/site-packages/django/forms/forms.py", line 274, in full_clean

    self._clean_form()

  File "/usr/lib/python2.7/site-packages/django/forms/forms.py", line 300, in _clean_form

    self.cleaned_data = self.clean()

  File "/usr/lib/python2.7/site-packages/django/contrib/auth/forms.py", line 189, in clean

    password=password)

  File "/usr/lib/python2.7/site-packages/django/contrib/auth/__init__.py", line 49, in authenticate

    user = backend.authenticate(**credentials)

  File "/usr/lib/python2.7/site-packages/reviewboard/accounts/backends.py", line 749, in authenticate

    import ldap

[Christian Hammond]

To check, was Apache restarted after installing the new module?

Christian

--
Supercharge your Review Board with Power Pack: https://www.reviewboard.org/powerpack/
Want us to host Review Board for you? Check out RBCommons: https://rbcommons.com/
Happy user? Let us know! https://www.reviewboard.org/users/
---
You received this message because you are subscribed to the Google Groups "reviewboard" group.
To unsubscribe from this group and stop receiving emails from it, send an email to reviewboard+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

[Alfred von Campe]

On Aug 15, 2016, at 18:03, Christian Hammond <chri...@beanbaginc.com> wrote:

To check, was Apache restarted after installing the new module?

No, but it occurred to me to do that just before I saw your post, and now it’s working!  It appears to be a bit slower than our previous instance, but that may be due to the hardware (or rather VM) that it’s running on.  I haven’t checked the specs of the VM that was provided by our IT department, but I will be comparing the two systems tomorrow.

Thanks for all your help,

Alfred

[Christian Hammond]

Is memcached properly set up, and Review Board configured to use it? That will make a huge difference in performance.

You can verify that it's being used by clicking Server Cache in the left sidebar of the admin UI. It should be reporting uptime, usage stats, etc., and that it's using memcached.

Christian

-- 
Christian Hammond

President/CEO of Beanbag

Makers of Review Board