Problem upgrade 0.5.2 --> 0.7

[Maximilian Schmidt]

Hi,
I am facing a problem while updating sumatra from 0.5.2 to 0.7.3.
I did `smt export` with 0.5.2, upgraded to 0.7.3 and then ran 'smt upgrade'. Then I encountered an error with the following traceback:

Traceback (most recent call last):
  File "/home/schmidt/.local/bin/smt", line 31, in <module>
    main(sys.argv[2:])
  File "/home/schmidt/.local/lib/python2.7/site-packages/sumatra/commands.py", line 627, in upgrade
    project.record_store.clear()
  File "/home/schmidt/.local/lib/python2.7/site-packages/sumatra/recordstore/django_store/__init__.py", line 306, in clear
    db_config.configure()
  File "/home/schmidt/.local/lib/python2.7/site-packages/sumatra/recordstore/django_store/__init__.py", line 123, in configure
    self._create_databases()
  File "/home/schmidt/.local/lib/python2.7/site-packages/sumatra/recordstore/django_store/__init__.py", line 112, in _create_databases
    management.call_command('migrate', database=label, verbosity=0)
  File "/home/schmidt/.local/lib/python2.7/site-packages/django/core/management/__init__.py", line 120, in call_command
    return command.execute(*args, **defaults)
  File "/home/schmidt/.local/lib/python2.7/site-packages/django/core/management/base.py", line 445, in execute
    output = self.handle(*args, **options)
  File "/home/schmidt/.local/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 222, in handle
    executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
  File "/home/schmidt/.local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 110, in migrate
    self.apply_migration(states[migration], migration, fake=fake, fake_initial=fake_initial)
  File "/home/schmidt/.local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 148, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/home/schmidt/.local/lib/python2.7/site-packages/django/db/migrations/migration.py", line 115, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/home/schmidt/.local/lib/python2.7/site-packages/django/db/migrations/operations/models.py", line 59, in database_forwards
    schema_editor.create_model(model)
  File "/home/schmidt/.local/lib/python2.7/site-packages/django/db/backends/base/schema.py", line 286, in create_model
    self.execute(sql, params or None)
  File "/home/schmidt/.local/lib/python2.7/site-packages/django/db/backends/base/schema.py", line 111, in execute
    cursor.execute(sql, params)
  File "/home/schmidt/.local/lib/python2.7/site-packages/django/db/backends/utils.py", line 79, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/home/schmidt/.local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/home/schmidt/.local/lib/python2.7/site-packages/django/db/utils.py", line 97, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/home/schmidt/.local/lib/python2.7/site-packages/django/db/backends/utils.py", line 62, in execute
    return self.cursor.execute(sql)
  File "/home/schmidt/.local/lib/python2.7/site-packages/django/db/backends/sqlite3/base.py", line 316, in execute
    return Database.Cursor.execute(self, query)
django.db.utils.OperationalError: table "django_content_type" already exists

My django version is 1.8.5.
What can I do?
Best,
Maximilian

-- 
Maximilian Schmidt
Institute of Neuroscience and Medicine (INM-6)
Computational and Systems Neuroscience
Institute for Advanced Simulation (IAS-6)       
Theoretical Neuroscience        
Juelich Research Center and JARA
Juelich, Germany

tel.: +49 2461 61-9472

max.s...@fz-juelich.de

------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------

[Maximilian Schmidt]

Any idea?

--
You received this message because you are subscribed to the Google Groups "sumatra-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sumatra-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

-- 
Maximilian Schmidt
Institute of Neuroscience and Medicine (INM-6)
Institute for Advanced Simulation (IAS-6)
JARA BRAIN Institute I
Juelich Research Centre
Juelich, Germany


tel.: +49 2461 61-9472

max.s...@fz-juelich.de

[Andrew Davison]

Hi Max,

Sorry for the delay in replying - I had a grant deadline which has taken up a lot of time.

I've just made a new release (0.7.4), which should fix the problem. Please take a backup copy of the SQLite file containing your records before trying the upgrade again (just in case - Sumatra also makes a backup - but better to be safe than sorry).

 
Cheers,

Andrew

On Friday, 16 October 2015 08:58:07 UTC+2, mschmidt wrote:

Any idea?

On 08.10.2015 10:52, Maximilian Schmidt wrote:

Hi,
I am facing a problem while updating sumatra from 0.5.2 to 0.7.3.
I did `smt export` with 0.5.2, upgraded to 0.7.3 and then ran 'smt upgrade'. Then I encountered an error with the following traceback:

Traceback (most recent call last):
  File "/home/schmidt/.local/bin/smt", line 31, in <module>
    main(sys.argv[2:])
  File "/home/schmidt/.local/lib/python2.7/site-packages/sumatra/commands.py", line 627, in upgrade
    project.record_store.clear()
  File "/home/schmidt/.local/lib/python2.7/site-packages/sumatra/recordstore/django_store/__init__.py", line 306, in clear
    db_config.configure()

...

  File "/home/schmidt/.local/lib/python2.7/site-packages/django/db/backends/sqlite3/base.py", line 316, in execute
    return Database.Cursor.execute(self, query)
django.db.utils.OperationalError: table "django_content_type" already exists

[Maximilian Schmidt]

Hi,
thanks for your reply and sorry for pressuring you, it's just that I can't run any simulation right now, because of the sumatra problem.
I have run into several other problems and I seem to be stuck with incompatible python packages.
So, I installed 0.7.4, run "smt upgrade", but then I got "Error: project must have been exported (with the original version of Sumatra) before upgrading."
Though I had done that (a week ago), I then downgraded to the original version (0.5.2), but now nothing works anymore.
I also downgrade gitpython to 0.2.0-beta1, because that was the first error.
Now I keep getting some django errors, though I have already downgraded to Django 1.5.0, and also tried 1.3.1.

With Django 1.3.1, I currently get this trace:

Traceback (most recent call last):
  File "/home/schmidt/.local/bin/smt", line 31, in <module>
    main(sys.argv[2:])

  File "/home/schmidt/.local/lib/python2.7/site-packages/sumatra/commands.py", line 355, in list
    print project.format_records(tags=tags, mode=options.mode, format=options.format)
  File "/home/schmidt/.local/lib/python2.7/site-packages/sumatra/projects.py", line 228, in format_records
    records = self.record_store.list(self.name, tags)
  File "/home/schmidt/.local/lib/python2.7/site-packages/sumatra/recordstore/django_store/__init__.py", line 212, in list
    db_records = self._manager.filter(project__id=project_name)
  File "/home/schmidt/.local/lib/python2.7/site-packages/sumatra/recordstore/django_store/__init__.py", line 138, in _manager
    models = self._get_models()
  File "/home/schmidt/.local/lib/python2.7/site-packages/sumatra/recordstore/django_store/__init__.py", line 123, in _get_models
    import models
  File "/home/schmidt/.local/lib/python2.7/site-packages/sumatra/recordstore/django_store/models.py", line 9, in <module>
    import tagging.fields
  File "/home/schmidt/.local/lib/python2.7/site-packages/tagging/fields.py", line 9, in <module>
    from tagging.models import Tag
  File "/home/schmidt/.local/lib/python2.7/site-packages/tagging/models.py", line 6, in <module>
    from django.utils.encoding import smart_text
ImportError: cannot import name smart_text

Is there a way out of this? I do have an ".smt_backup_..." folder with the files "project", "project_export.json" and "records_export.json", but sumatra 0.7.4. does not seem to recognize it.

Thanks,
Max

[Andrew Davison]

On 16 Oct 2015, at 15:52, Maximilian Schmidt <max.s...@fz-juelich.de> wrote:
>
> Hi,
> thanks for your reply and sorry for pressuring you, it's just that I can't run any simulation right now, because of the sumatra problem.
> I have run into several other problems and I seem to be stuck with incompatible python packages.
> So, I installed 0.7.4, run "smt upgrade", but then I got "Error: project must have been exported (with the original version of Sumatra) before upgrading."

> Though I had done that (a week ago), I then downgraded to the original version (0.5.2), but now nothing works anymore.
> I also downgrade gitpython to 0.2.0-beta1, because that was the first error.
> Now I keep getting some django errors, though I have already downgraded to Django 1.5.0, and also tried 1.3.1.

> ...

> Is there a way out of this? I do have an ".smt_backup_..." folder with the files "project", "project_export.json" and "records_export.json", but sumatra 0.7.4. does not seem to recognize it.

Sumatra is looking for a file ".smt/project_export.json" but doesn't find it. Try renaming ".smt_backup_..." to ".smt" (keep an extra copy, though) then run "smt upgrade" again.

Cheers,

Andrew

[Maximilian Schmidt]

Hi,
thanks, now I am 1 step further. However, running 'smt upgrade' produced this output:

NEST could not be found. Please supply the path to the /home/schmidt/nest/10kcollaps_gsd.build/install/bin/nest executable.
Upgrading from a Sumatra version which did not have the --plain configuration
option. After this upgrade, arguments to 'smt run' of the form 'name=value'
will continue to overwrite default parameter values, but this is now
configurable. If it is desired that they should be passed straight through
to the program, run the command 'smt configure --plain' after this upgrade.

/home/schmidt/.local/lib/python2.7/site-packages/sumatra/programs.py:65: Warning: NEST could not be found. Please supply the path to the /homea/jinb33/jinb3309/nest_thread_perf/ins/bin/nest executable.
  warnings.warn(errmsg)

/home/schmidt/.local/lib/python2.7/site-packages/sumatra/programs.py:65: Warning: NEST could not be found. Please supply the path to the /users/schmidt/nest/nest_bold_threaded_perf.build/install/bin/nest executable.
  warnings.warn(errmsg)

Traceback (most recent call last):
  File "/home/schmidt/.local/bin/smt", line 31, in <module>
    main(sys.argv[2:])

  File "/home/schmidt/.local/lib/python2.7/site-packages/sumatra/commands.py", line 630, in upgrade
    project.record_store.import_(project.name, f.read())
  File "/home/schmidt/.local/lib/python2.7/site-packages/sumatra/recordstore/base.py", line 77, in import_
    records = serialization.decode_records(content)
  File "/home/schmidt/.local/lib/python2.7/site-packages/sumatra/recordstore/serialization.py", line 161, in decode_records
    return [build_record(data) for data in json.loads(content)]
  File "/home/schmidt/.local/lib/python2.7/site-packages/sumatra/recordstore/serialization.py", line 80, in build_record
    repository = repos_cls(rdata["url"])
  File "/home/schmidt/.local/lib/python2.7/site-packages/sumatra/versioncontrol/_git.py", line 145, in __init__
    self.upstream = self.upstream or self._get_upstream()
  File "/home/schmidt/.local/lib/python2.7/site-packages/sumatra/versioncontrol/_git.py", line 181, in _get_upstream
    if self.exists:
  File "/home/schmidt/.local/lib/python2.7/site-packages/sumatra/versioncontrol/_git.py", line 150, in exists
    self._repository
  File "/home/schmidt/.local/lib/python2.7/site-packages/sumatra/versioncontrol/_git.py", line 159, in _repository
    self.__repository = git.Repo(self.url)
  File "/usr/local/lib/python2.7/dist-packages/git/repo/base.py", line 121, in __init__
    raise NoSuchPathError(epath)
git.exc.NoSuchPathError: /users/schmidt/model-june

where "/users/schmidt/model-june" is the repository location on our local cluster.
After that, I can run smt list, but returns an empty list of recordings, the same happens with "smtweb".
I had a similar error over 2 years ago, back then, you recommended cloning the repository and installing from there, because you had fixed a bug, but I assume that this can't be the problem then anymore.
The mail exchange is here: https://groups.google.com/forum/#!searchin/sumatra-users/$2Fusers$2Fschmidt$2F/sumatra-users/64b0Cf9zyuM/0R3TwaRy9BIJ

What can I do?
Thanks,
Max

-- 
Maximilian Schmidt
Institute of Neuroscience and Medicine (INM-6)
Computational and Systems Neuroscience
Institute for Advanced Simulation (IAS-6)       
Theoretical Neuroscience        
Juelich Research Center and JARA
Juelich, Germany

tel.: +49 2461 61-9472

max.s...@fz-juelich.de

[Andrew Davison]

Which version of gitpython are you using?

This is very strange, because the line "self.__repository = git.Repo(self.url)" is wrapped in a try..except block which should catch the NoSuchPathError.

[Andrew Davison]

By the way, if this is holding up your work, I suggest you create a fresh Sumatra project with 0.7.4. with the same name but in a different directory and with a different filename for the SQLite record store, and use that while we're trying to fix the upgrade problem.

Merging the new and old stores should be straightforward ("smt sync" command), once they're both using the same Sumatra version.

[Maximilian Schmidt]

I am now using gitpython 0.3.5, because with 1.0.1, sumatra was
complaining that this version is too old, and that it requires at least
0.3.5. I guess this is the same error as in the mail from Eli Alster in May.

[Andrew Davison]

On Friday, 16 October 2015 17:23:02 UTC+2, mschmidt wrote:

I am now using gitpython 0.3.5, because with 1.0.1, sumatra was
complaining that this version is too old, and that it requires at least
0.3.5. I guess this is the same error as in the mail from Eli Alster in May.

The gitpython error problem was fixed some time ago. I am using 1.0.1 on my machine, and I think Travis CI uses the same version to run the test suite.

Could it be that your system is finding an old version of Sumatra?

 

[Maximilian Schmidt]

Hi,
so I repeated everything from the beginning and now it works. I guess, I must have confused some libraries in the whole processing, or not repeated all steps in the previous trails, sorry for that.
Thanks a lot for your help!
Best,
Max