Issues running upgrade-db on 1.6.0, 1.7.1 (PostgreSQL)

[Stephen Price]

I'm trying to test 1.7.1 in staging. I first tried a simple "pip install --upgrade rhodecode", but running "paster upgrade-db" gave me this error:

$ paster upgrade-db production.01.ini
You are about to perform database upgrade, make sure You backed up your database before. Continue ? [y/n]y
Traceback (most recent call last):
File "/srv/rhodecode/venv/bin/paster", line 8, in <module>
load_entry_point('PasteScript==1.7.5', 'console_scripts', 'paster')()
File "/srv/rhodecode/venv/lib/python2.6/site-packages/paste/script/command.py", line 104, in run
invoke(command, command_name, options, args[1:])
File "/srv/rhodecode/venv/lib/python2.6/site-packages/paste/script/command.py", line 143, in invoke
exit_code = runner.run(args)
File "/srv/rhodecode/venv/lib/python2.6/site-packages/rhodecode/lib/utils.py", line 697, in run
return super(BasePasterCommand, self).run(args[1:])
File "/srv/rhodecode/venv/lib/python2.6/site-packages/paste/script/command.py", line 238, in run
result = self.command()
File "/srv/rhodecode/venv/lib/python2.6/site-packages/rhodecode/lib/dbmigrate/__init__.py", line 59, in command
dbmanage.upgrade()
File "/srv/rhodecode/venv/lib/python2.6/site-packages/rhodecode/lib/db_manage.py", line 145, in upgrade
api.version_control(db_uri, repository_path, curr_version)
File "<string>", line 2, in version_control
File "/srv/rhodecode/venv/lib/python2.6/site-packages/rhodecode/lib/dbmigrate/migrate/versioning/util/__init__.py", line 159, in with_engine
return f(*a, **kw)
File "/srv/rhodecode/venv/lib/python2.6/site-packages/rhodecode/lib/dbmigrate/migrate/versioning/api.py", line 250, in version_control
ControlledSchema.create(engine, repository, version)
File "/srv/rhodecode/venv/lib/python2.6/site-packages/rhodecode/lib/dbmigrate/migrate/versioning/schema.py", line 139, in create
table = cls._create_table_version(engine, repository, version)
File "/srv/rhodecode/venv/lib/python2.6/site-packages/rhodecode/lib/dbmigrate/migrate/versioning/schema.py", line 185, in _create_table_version
result = engine.execute(s, repository_id=repository.id)
File "/srv/rhodecode/venv/lib/python2.6/site-packages/sqlalchemy/engine/base.py", line 2446, in execute
return connection.execute(statement, *multiparams, **params)
File "/srv/rhodecode/venv/lib/python2.6/site-packages/sqlalchemy/engine/base.py", line 1449, in execute
params)
File "/srv/rhodecode/venv/lib/python2.6/site-packages/sqlalchemy/engine/base.py", line 1584, in _execute_clauseelement
compiled_sql, distilled_params
File "/srv/rhodecode/venv/lib/python2.6/site-packages/sqlalchemy/engine/base.py", line 1698, in _execute_context
context)
File "/srv/rhodecode/venv/lib/python2.6/site-packages/sqlalchemy/engine/base.py", line 1691, in _execute_context
context)
File "/srv/rhodecode/venv/lib/python2.6/site-packages/sqlalchemy/engine/default.py", line 331, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.DataError: (DataError) invalid input syntax for integer: "rhodecode_db_migrations"
LINE 3: WHERE db_migrate_version.repository_id = E'rhodecode_db_migr...
^
'SELECT db_migrate_version.repository_id, db_migrate_version.repository_path, db_migrate_version.version \nFROM db_migrate_version \nWHERE db_migrate_version.repository_id = %(repository_id)s' {'repository_id': 'rhodecode_db_migrations'}

Next, I tried uninstalling rhodecode and installing 1.6.0 first (via pip), same error. Then, I tried upgrading sqlalchemy and psycopg2 and installing rhodecode, same error. Here's the output of "pip freeze":

$ pip freeze

Babel==0.9.6

Beaker==1.6.4

FormEncode==1.2.4

Mako==0.7.3

Markdown==2.2.1

MarkupSafe==0.18

Paste==1.7.5.1

PasteDeploy==1.5.0

PasteScript==1.7.5

Pygments==1.6

Pylons==1.0

RhodeCode==1.6.0

Routes==1.13

SQLAlchemy==0.7.10

Tempita==0.5.1

WebError==0.10.3

WebHelpers==1.3

WebOb==1.0.8

WebTest==1.4.3

Whoosh==2.4.1

amqplib==1.0.2

anyjson==0.3.3

argparse==1.2.1

celery==2.2.10

decorator==3.4.0

docutils==0.8.1

dulwich==0.8.7

git-remote-helpers==0.1.0

importlib==1.0.2

keyring==1.2.2

kombu==1.5.1

mercurial==2.5.4

mercurial-keyring==0.5.4

mock==1.0.1

nose==1.3.0

psycopg2==2.5.1

py-bcrypt==0.3

pyparsing==1.5.7

python-dateutil==1.5

python-ldap==2.4.10

repoze.lru==0.6

simplejson==2.5.2

unittest2==0.5.1

waitress==0.8.2

Any advice? Let me know if I should provide more info. 

[Stephen Price]

Running on CentOS 6 with PostgreSQL 8.4.13.

[Marcin Kuzminski]

Hi,

It's super odd, the column `repository_id` by schema definition is a string.

 repository_id = Column('repository_id', String(250), primary_key=True)

Could you check after installing RhodeCode what type it is created in your database ?

Cheers

Marcin Kuzminski

--
You received this message because you are subscribed to the Google Groups "rhodecode" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rhodecode+...@googlegroups.com.
To post to this group, send email to rhod...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Stephen Price]

I started with RhodeCode 1.5.4, and it had previously used the default SQLite db. I migrated it to Postgres using the yaml_db gem, and hadn't had any problems upgrading from previous versions.

Here's what the table currently looks like:

rhodecode=# \d+ db_migrate_version

                                                Table "public.db_migrate_version"

     Column      |  Type   |                                 Modifiers                                  | Storage  | Description

-----------------+---------+----------------------------------------------------------------------------+----------+---

 repository_id   | integer | not null default nextval('db_migrate_version_repository_id_seq'::regclass) | plain    |

 repository_path | text    |                                                                            | extended |

 version         | integer |                                                                            | plain    |

Indexes:

    "db_migrate_version_pkey" PRIMARY KEY, btree (repository_id)

    "sqlite_autoindex_db_migrate_version_1" UNIQUE, btree (repository_id)

Has OIDs: no

[Marcin Kuzminski]

In 1.7.X migrations work little bit different. I'm really surprised that it had worked before. Because it shouldn't. Maybe some changed in sqlalchemy triggered that error now.

Please fix the column type, and try again.

cheers

Marcin Kuzminski

[Stephen Price]

Ok, I started with a backup from production (running 1.5.4), then ran "alter table db_migrate_version alter column repository_id type varchar(250);" Now upgrade-db gives me a new error:

sqlalchemy.exc.ProgrammingError: (ProgrammingError) column "repo_type" of relation "repositories" already exists

 '\nALTER TABLE repositories ADD repo_type VARCHAR' {}

I did see this message at the beginning though:

*** CURRENT DATABASE IS NOT UNDER VERSION CONTROL. SETTING AS VERSION 1 ***
***************************************************************************
*** PERFORMING UPGRADE STEP 0 ***
*********************************
*** ATTEMPTING TO DO DATABASE UPGRADE FROM VERSION 1 TO VERSION 11 ***
**********************************************************************

Any chance messing with that column did this?

[Marcin Kuzminski]

Looks like you lost current version of db schema, it should be set to version 10 for RhodeCode 1.5.4.

In the first place you should update the version column to 10 in this table. But if that would be missing non of the upgrades before should have worked before.

Secondly upgrading to 1.7.1 should end in version 14 not 11, so it looks like it tried to upgrade to 1.6.0 not 1.7.1 ?

Marcin Kuzminski

[Stephen Price]

I was following the advice in the upgrade instructions to upgrade to the next major version first, so yes, I upgraded to 1.6.0 first.

Are you saying after changing the column to varchar(250) that I should manually update it to '10'?

[Stephen Price]

So, the entry in the 'version' column is still '10' when I change the 'repository_id' column to varchar(250), but my guess is that update_db can't look up this row since the value changed (from an integer 0 to '0'), and assumes it's the default (version 1).

[Marcin Kuzminski]

might be, did you try to change it to integer since the version column should be integer ?

Marcin Kuzminski

[Stephen Price]

No, the version column is still integer, I didn't change it. I only changed the 'repository_id' column from integer to varchar(250).

[Stephen Price]

Finally got past this issue. I had to both alter the table so that 'repository_id' was varchar(250), and then set repository_id = 'rhodecode_db_migrations' in the single row in that table. After that the upgrade finished, and I could also upgrade to 1.7.1.