Incompatible Database Schema Version

[Sam Shepherd]

Hi all, 

I've encountered an error when updating my aiida installation (updated from 1.6.2 to 1.6.5 using pip), I'm able to use the command line as required when the command itself doesn't involve any interaction with the database. However, when I run any command that does after my installation, I am met with the following output...

Database schema version `1.0.48` is incompatible with the required schema version `1.0.45`.

To migrate the database schema version to the current one, run the following command:

    verdi -p samshepherd database migrate

Initially, I thought this made sense and ran the command, which then provided...

Cannot migrate the database version from `1.0.48` to `1.0.45`.

The database version is ahead of the version of the code and downgrades of the database are not supported.

I want to understand how I managed to cause this, so as to hopefully avoid it in the future. I would like to preface this by saying I followed the update procedure to the letter, and have made a backup of both my repository and database, so immediate data loss is not a concern. I'm more concerned about the ability which I'll be able to interact with the results of these older calculations if I can't load up the database. 

My current belief is that I waited too long between updates and somehow that caused an inconsistency with the database. There are a number of things I think might be a possible solution to my particular problem that I would prefer more than having to resort to some extreme method. 

1. Reroll the update back to 1.6.2: I was already getting strange inconsistencies with AiiDA submitted calculations and I would prefer to update to the newest version anyway for maintenance.
2. Edit the database: I make this comment with no SQL or PSQL knowledge, but rather if it is an option anyone else would be aware of to simply 'edit' the version of the database, or recreate it in the proper version within a new database.
3. Do a clean AiiDA reinstall: I don't see what this would accomplish for trying to migrate a database over to the newer version if I'm trying that same version currently, but I may be looking at that too simplistically.

Any help on this matter is greatly appreciated, and if anything additional is required to help diagnose the problem, please let me know. 

Best, 

Sam Shepherd

 

[Sebastiaan Huber]

Hi Sam,

This is a weird situation indeed.
The migration version that comes with 1.6.5 is `1.0.45` and that is the latest migration version that is released.
The migration `1.0.48`, which your database seems to be at currently, has not been released and is only present on the `develop` branch.
This branch will be released in the future as AiiDA v2.0.

Is it possible that you have checked out the `aiida-core` git repository at some point and `pip install`ed from the repository?
That is the only explanation that I can come up with how you got the migrations from `develop`.

To solve your situation, you have essentially two options:

1) Properly install `aiida-core==1.6.5` and revert your database.
2) Keep your migrated data and continue to work off of develop.

The latter option is probably the most straightforward but definitely not recommended.
The `develop` branch is not guaranteed to be stable, so if you are using AiiDA for production work, I would not use it.
Still, if you prefer this option, I added instructions at the bottom of this email for reference.

To revert your data, you should recreate the Postgres database from your backup and then make sure you have the correct AiiDA version installed.
First, make sure you have the correct version of AiiDA installed:

    pip uninstall aiida-core
    pip install aiida-core==1.6.5

Then you need to drop and recreate the PostgreSQL database.
For this you need the database dump that you say you have created before you migrated.
So I am assuming you have a file somewhere that is the result of a `pg_dump` call, probably a `.psql` file.
If you don't have this file, there is nothing we can do unfortunately, otherwise you can continue reading the instructions below.

In restoring the database, there is nothing AiiDA specific and you can find many resources for restoring a Postgres database online.
But I will give a rundown of it here for convenience.
Note though that if you don't have any experience with this, it may be a bit complicated.

To drop and recreate the database you need the name of the database and the user and password you used, which you can find by running `verdi profile show samshepherd`.
It should show something like:

aiidadb_backend         django
aiidadb_engine          postgresql_psycopg2
aiidadb_host            localhost
aiidadb_name            aiida_dev_django
aiidadb_pass            aiida_password
aiidadb_user            aiida_user

Then you need to run the `psql` command as the `postgres` user.
You first will need to change to that `postgres` user

    sudo su - postgres

Then you run

    psql

which should show you a prompt.
Then you drop the database (WARNING: this will delete your data, so make sure you have a proper backup!)

    DROP DATABASE aiida_dev_django;

Replace the name with the value you got from

    CREATE DATABASE aiida_dev_django WITH OWNER aiida_user;

This should create a new empty database
Then you exit the shell by typing `\q`.
Then you restore the content of the database from your backup by running:

    psql -U aiida_user -h localhost -d aiida_dev_django -W < your_backup_file.psql

Here you replace `aiida_user` with the user name of your database and `aiida_dev_django` with the database name.
You should be prompted for the database password, which you should have gotten from the `verdi profile show`.
The database should now be restored.
You can check that everything is in order with `verdi status`.
I hope this will help you to restore your environment to working order.


HTH,

SPH


To install the develop branch such that you can continue using your already migrated data, simply do the following in your loaded virtual environment.
Again, this is not recommended for production work.

    pip uninstall aiida-core
    git clone https://github.com/aiidateam/aiida-core
    cd aiida-core
    pip install -e .

This will install the latest develop version in your environment.
Note that there maybe other changes that require some action before it works.
For example, you may have to apply another migration.

--
AiiDA is supported by the NCCR MARVEL (http://nccr-marvel.ch/), funded by the Swiss National Science Foundation, and by the European H2020 MaX Centre of Excellence (http://www.max-centre.eu/).
 
Before posting your first question, please see the posting guidelines at http://www.aiida.net/?page_id=356 .
---
You received this message because you are subscribed to the Google Groups "aiidausers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aiidausers+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/aiidausers/06929c1a-e222-4784-b808-171ae2bb581bn%40googlegroups.com.

[Sam Shepherd]

Hi Sebastian, 

First, thanks for your quick reply.

Unfortunately, I believe you are right and somewhere along the lines I installed a local version of aiida from the develop branch of the github, potentially in updating that I may have as you say actually downgraded to a previous stable version using pip. 

I tried to recreate the database, however the same problem occurred regarding the database version, so in the end I have updated to the develop branch of aiida present on github. It's not ideal, but it worked and allowed me to migrate the database over. 

Thanks again for the help!

Sam