Postgres Database Schema modification fails on upgrade from 1.1.0 to 1.2.1

[Chris Lewis]

Hello,

We are currently using Transifex 1.1.0 CE with an external Postgres
DB. When attempting to upgrade from to version 1.2.1 (following steps
in docs) we get:

root@transifex:/usr/local/lib/python2.6/dist-packages/transifex-1.2.1-
py2.6.egg/transifex# ./manage.py syncdb --noinput
Syncing...
Creating tables ...
Installing custom SQL ...
Installing indexes ...
No fixtures found.

Synced:
> django.contrib.admin
> django.contrib.auth
> django.contrib.comments
> django.contrib.contenttypes
> django.contrib.flatpages
> django.contrib.markup
> django.contrib.sessions
> django.contrib.messages
> django.contrib.sites
> django.contrib.admindocs
> django.contrib.staticfiles
> notification
> django_filters
> django_sorting
> south
> tagging
> tagging_autocomplete
> pagination
> piston
> ajax_select
> threadedcomments
> authority
> haystack
> social_auth
> django_addons
> compressor
> charts
> datastores
> helptext
> lotte
> timeline
> trans_memory
> txcron
> widgets

Not synced (use migrations):
- userena
- guardian
- easy_thumbnails
- transifex.languages
- transifex.projects
- transifex.actionlog
- transifex.txpermissions
- transifex.teams
- transifex.resources
- transifex.storage
- copyright
- gtranslate
- jsonmap
- locks
- priorities
- suggestions
- txapps
- watches
- webhooks
- autofetch
- transifex.releases
- transifex.txcommon
(use ./manage.py migrate to migrate these)
root@transifex:/usr/local/lib/python2.6/dist-packages/transifex-1.2.1-
py2.6.egg/transifex# python manage.py migrate copyright 0001 --fake
- Soft matched migration 0001 to 0001_initial.
Running migrations for copyright:
- Nothing to migrate.
root@transifex:/usr/local/lib/python2.6/dist-packages/transifex-1.2.1-
py2.6.egg/transifex# python manage.py migrate --ignore-ghost-
migrations --noinput
Running migrations for userena:
- Nothing to migrate.
- Loading initial data for userena.
No fixtures found.
Running migrations for guardian:
- Nothing to migrate.
- Loading initial data for guardian.
No fixtures found.
Running migrations for easy_thumbnails:
- Nothing to migrate.
- Loading initial data for easy_thumbnails.
No fixtures found.
Running migrations for languages:
- Nothing to migrate.
- Loading initial data for languages.
No fixtures found.
Running migrations for projects:
- Migrating forwards to 0006_finalize_source_language_field.
> projects:0005_set_source_language_from_resources
Error in migration: projects:0005_set_source_language_from_resources
Traceback (most recent call last):
File "manage.py", line 21, in <module>
execute_manager(settings)
File "/usr/local/lib/python2.6/dist-packages/Django-1.3.1-py2.6.egg/
django/core/management/__init__.py", line 438, in execute_manager
utility.execute()
File "/usr/local/lib/python2.6/dist-packages/Django-1.3.1-py2.6.egg/
django/core/management/__init__.py", line 379, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python2.6/dist-packages/Django-1.3.1-py2.6.egg/
django/core/management/base.py", line 191, in run_from_argv
self.execute(*args, **options.__dict__)
File "/usr/local/lib/python2.6/dist-packages/Django-1.3.1-py2.6.egg/
django/core/management/base.py", line 220, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python2.6/dist-packages/South-0.7.5-py2.6.egg/
south/management/commands/migrate.py", line 107, in handle
ignore_ghosts = ignore_ghosts,
File "/usr/local/lib/python2.6/dist-packages/South-0.7.5-py2.6.egg/
south/migration/__init__.py", line 219, in migrate_app
success = migrator.migrate_many(target, workplan, database)
File "/usr/local/lib/python2.6/dist-packages/South-0.7.5-py2.6.egg/
south/migration/migrators.py", line 235, in migrate_many
result = migrator.__class__.migrate_many(migrator, target,
migrations, database)
File "/usr/local/lib/python2.6/dist-packages/South-0.7.5-py2.6.egg/
south/migration/migrators.py", line 310, in migrate_many
result = self.migrate(migration, database)
File "/usr/local/lib/python2.6/dist-packages/South-0.7.5-py2.6.egg/
south/migration/migrators.py", line 133, in migrate
result = self.run(migration)
File "/usr/local/lib/python2.6/dist-packages/South-0.7.5-py2.6.egg/
south/migration/migrators.py", line 107, in run
return self.run_migration(migration)
File "/usr/local/lib/python2.6/dist-packages/South-0.7.5-py2.6.egg/
south/migration/migrators.py", line 81, in run_migration
migration_function()
File "/usr/local/lib/python2.6/dist-packages/South-0.7.5-py2.6.egg/
south/migration/migrators.py", line 57, in <lambda>
return (lambda: direction(orm))
File "/usr/local/lib/python2.6/dist-packages/transifex-1.2.1-
py2.6.egg/transifex/projects/migrations/
0005_set_source_language_from_resources.py", line 14, in forwards
resource = Resource.objects.filter(project=project)[0]
File "/usr/local/lib/python2.6/dist-packages/Django-1.3.1-py2.6.egg/
django/db/models/query.py", line 190, in __getitem__
return list(qs)[0]
File "/usr/local/lib/python2.6/dist-packages/Django-1.3.1-py2.6.egg/
django/db/models/query.py", line 84, in __len__
self._result_cache.extend(self._iter)
File "/usr/local/lib/python2.6/dist-packages/Django-1.3.1-py2.6.egg/
django/db/models/query.py", line 273, in iterator
for row in compiler.results_iter():
File "/usr/local/lib/python2.6/dist-packages/Django-1.3.1-py2.6.egg/
django/db/models/sql/compiler.py", line 680, in results_iter
for rows in self.execute_sql(MULTI):
File "/usr/local/lib/python2.6/dist-packages/Django-1.3.1-py2.6.egg/
django/db/models/sql/compiler.py", line 735, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python2.6/dist-packages/Django-1.3.1-py2.6.egg/
django/db/backends/util.py", line 34, in execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python2.6/dist-packages/Django-1.3.1-py2.6.egg/
django/db/backends/postgresql_psycopg2/base.py", line 44, in execute
return self.cursor.execute(query, args)
django.db.utils.DatabaseError: column resources_resource.category does
not exist
LINE 1: ...urce"."last_update", "resources_resource"."slug",
"resources...
^

root@transifex:/usr/local/lib/python2.6/dist-packages/transifex-1.2.1-
py2.6.egg/transifex#

Looks a like it's expecting to find a column which isn't there.
The pg database being connected to is a restored backup of the current
1.2.0 system.

The system being used is Debian 6 (base install) with build-essential
python-dev python-setuptools python-imaging gettext intltool apache2
libapache2-mod-wsgi postgresql-client libpq-d

Transifex and psycopg2 were installed using Easy_install.

Have tried installing Transifex 1.1 and linking to restored backup
(which worked) then upgrading. Failed at the same point.

Thanks in advance for any help with this.

Best regards

Chris

[Apostolis Bessas]

Hi,

It seems there is a dependency among migrations missing.
Can you do a
./manage.py migrate resources
./manage.py migrate

and see if it works?

Regards,
Apostolis

--
Apostolis Bessas
Systems Engineer

https://www.transifex.com/

[Chris Lewis]

Hi Apostolis,

Thanks for your kind response.

It seemed to help (initially)

We ran:
python manage.py syncdb --noinput

python manage.py migrate copyright 0001 --fake

./manage.py migrate resources
python manage.py migrate --ignore-ghost-migrations --noinput
python manage.py txcreatenoticetypes
python manage.py txlanguages
python manage.py collectstatic
./manage.py runserver 192.168.122.141:8000

without error.

We attempted to log in as a user through the www interface.

We got an error:


Request Method: GET
Request URL: http://192.168.122.141:8000/
Django Version: 1.3.1
Exception Type: DatabaseError
Exception Value:

column notification_notice.user_id does not exist
LINE 1: ...RE ("notification_notice"."archived" = false AND "notificat...
^

Exception Location:
/usr/local/lib/python2.6/dist-packages/Django-1.3.1-py2.6.egg/django/db/backends/postgresql_psycopg2/base.py
in execute, line 44
Python Executable: /usr/bin/python
Python Version: 2.6.6


It still looks like the upgrade process has not worked correctly.

Thanks in advance for any further advice.

Regards

Chris

[Apostolis Bessas]

Hi Chris,

On Wed, May 23, 2012 at 7:55 PM, Chris Lewis
<chrispl...@googlemail.com> wrote:
>
> column notification_notice.user_id does not exist
> LINE 1: ...RE ("notification_notice"."archived" = false  AND "notificat...
>                                                             ^

This seems to be caused by a wrong version of django-notification.
Can you confirm that you have the 1.5.0 version installed?

$ pip freeze | grep django-notification
django-notification==0.1.5

Regards,
Apostolis

[Chris Lewis]

Hi Apostolis

Yes we have that version : django-notification==0.1.5

regards

Chris

[Apostolis Bessas]

The columns the table notification_notice should have are

id
user_id
message
notice_type_id
added
unseen
archived
on_site

It seems you miss the ``user_id`` one. Did you upgrade
django-notification recently? Anyway, the thing is that
django-notification does not provide a way to change the schema in the
database. So, you either need to add the column manually:

ALTER TABLE notification_notice ADD COLUMN user_id INTEGER NOT NULL
FOREIGN KEY auth_user(id);
CREATE INDEX notification_notice_user_id ON notification_notice(user_id);

or drop the table (and lose the notifications data) by connecting to
postgresql and executing the command

DROP TABLE notification_notice;

and run

./manage.py syncdb

again.

Disclaimer: I have not tried the SQL commands, so take a backup first :)

Regards,
Apostolis

On Thu, May 24, 2012 at 12:45 PM, Chris Lewis