UnicodeDecodeError: 'utf8' codec can't decode byte 0xbc in position 0: invalid start byte

[Michael Governale]

We ran into this issue twice this week. Going to review pages and repository config pages is giving a 500 error. The log messages are below.

The last time we deleted and recreated the repository configuration pages. This loses the old / active reviews and isn't ideal.

Any help or advice is appreciate. Thanks.

Review Board 2.5.7

Python warning:

	- /usr/lib/python2.7/site-packages/feedparser.py:364: DeprecationWarning: To avoid breaking existing software while fixing issue 310, a temporary mapping has been created from `updated_parsed` to `published_parsed` if `updated_parsed` doesn't exist. This fallback will be removed in a future version of feedparser. DeprecationWarning)

Repository configuration error:

- Exception thrown for user at http://reviewboard/admin/db/scmtools/repository/9/ 'utf8' codec can't decode byte 0xbc in position 0: invalid start byte Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 112, in get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/lib/python2.7/site-packages/django/contrib/admin/options.py", line 465, in wrapper return self.admin_site.admin_view(view)(*args, **kwargs) File "/usr/lib/python2.7/site-packages/django/utils/decorators.py", line 99, in _wrapped_view response = view_func(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/django/views/decorators/cache.py", line 52, in _wrapped_view_func response = view_func(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/django/contrib/admin/sites.py", line 198, in inner return view(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/django/utils/decorators.py", line 29, in _wrapper return bound_func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/django/utils/decorators.py", line 99, in _wrapped_view response = view_func(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/django/utils/decorators.py", line 25, in bound_func return func(self, *args2, **kwargs2) File "/usr/lib/python2.7/site-packages/django/db/transaction.py", line 371, in inner return func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/django/contrib/admin/options.py", line 1270, in change_view form = ModelForm(instance=obj) File "/usr/lib/python2.7/site-packages/reviewboard/scmtools/forms.py", line 431, in __init__ self._populate_repository_info_fields() File "/usr/lib/python2.7/site-packages/reviewboard/scmtools/forms.py", line 517, in _populate_repository_info_fields self.fields['password'].initial = self.instance.password File "/usr/lib/python2.7/site-packages/reviewboard/scmtools/models.py", line 217, in _get_password password = decrypt_password(password).decode('utf-8') File "/usr/lib64/python2.7/encodings/utf_8.py", line 16, in decode return codecs.utf_8_decode(input, errors, True) UnicodeDecodeError: 'utf8' codec can't decode byte 0xbc in position 0: invalid start byte

Review error:

 - Exception thrown for user at http://reviewboard/r/19/

'utf8' codec can't decode byte 0xa9 in position 0: invalid start byte
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 112, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/lib/python2.7/site-packages/reviewboard/accounts/decorators.py", line 25, in _check
    return view_func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/reviewboard/site/decorators.py", line 35, in _check
    return view_func(request, local_site=local_site, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/reviewboard/reviews/views.py", line 780, in review_detail
    'screenshots': screenshots,
  File "/usr/lib/python2.7/site-packages/reviewboard/reviews/context.py", line 99, in make_review_request_context
    upload_diff_form = UploadDiffForm(review_request, request=request)
  File "/usr/lib/python2.7/site-packages/reviewboard/reviews/forms.py", line 127, in __init__
    data, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/reviewboard/diffviewer/forms.py", line 41, in __init__
    if self.repository.get_scmtool().get_diffs_use_absolute_paths():
  File "/usr/lib/python2.7/site-packages/reviewboard/scmtools/models.py", line 231, in get_scmtool
    return cls(self)
  File "/usr/lib/python2.7/site-packages/reviewboard/scmtools/svn/__init__.py", line 69, in __init__
    credentials = repository.get_credentials()
  File "/usr/lib/python2.7/site-packages/reviewboard/scmtools/models.py", line 281, in get_credentials
    password = self.password
  File "/usr/lib/python2.7/site-packages/reviewboard/scmtools/models.py", line 217, in _get_password
    password = decrypt_password(password).decode('utf-8')
  File "/usr/lib64/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xa9 in position 0: invalid start byte

[Christian Hammond]

Hi,

We've seen this happen when manually manipulating the contents of these columns in the table. Was there any database work being done via script or manual SQL manipulation?

Basically, the code looks in the password column for a special character at the beginning (a tab). If present, the following data is expected to be encrypted. If not, it's a plain text password, which will then be encrypted.

What can happen is if the tab was ever dropped, it would have seen the content as plain text and tried to re-encrypt it. The next decrypt would have then returned the binary content from the previous encryption, instead of valid UTF-8 content. This triggers the errors you see below.

If this is happening occasionally, then these tables are being manually manipulated in some way that corrupts the field. Can you think of what may be running or what SQL work was done that would cause that?

A common one is dumping the entries to the screen using SQL and then copy/pasting them back into the database (usually another database).

You shouldn't delete the repository entries, though. Once I know what's going on, I can give you a solution for fixing up the passwords.

Christian

--
Supercharge your Review Board with Power Pack: https://www.reviewboard.org/powerpack/
Want us to host Review Board for you? Check out RBCommons: https://rbcommons.com/
Happy user? Let us know! https://www.reviewboard.org/users/
---
You received this message because you are subscribed to the Google Groups "reviewboard" group.
To unsubscribe from this group and stop receiving emails from it, send an email to reviewboard...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

-- 
Christian Hammond

President/CEO of Beanbag

Makers of Review Board

[Christian Hammond]

Another thing to check is that your SECRET_KEY in settings_local.py hasn't changed. If you're regenerating your site directory from scratch, this will happen.

Christian 

[sha...@burst.com]

This is happening with me too. I installed a dockerized RB (https://github.com/ikatson/docker-reviewboard). Every time I kill and restart the docker (using 'docker stop'), I see the error mentioned above when click on any reviews or the svn repository link.

I am not modifying the DB manually, any SECRET_KEY in settings_local.py doesn't change too. This looks like a real bug in RB. I am using version 2.5.7.

Thanks,

Shashi

[Christian Hammond]

Can you try creating a dummy repository entry and using "test" as the password? Then attempt to repeat the error for that dummy repository. If it happens, dump the SQL for that repository entry and see if the beginning of it starts with a tab or not.

Christian