Need help in Migrating data from old instance of OTM1 to new code.

[Ulhas Shah]

Hi,

We are running a old instance of OTM and now we want to migrate it to new latest code base.

I have setup a new instance also.

Its a clean instance.

How can I migrate the data only from old site to newly setup instance based on latest code?

What process do I need to do to transfer only data from the database?

I tried to replace the database but it gave me schema error..

Can any one help me with this?

---

Ulhas Shah

[Justin Walgran]

There is an migrator that imports fixtures dumped from OTM1 into an OTM2 instance

https://github.com/OpenTreeMap/otm-core/tree/develop/opentreemap/otm1_migrator

There is an OTM1 management command to avoid memory errors when dumping large fixtures.

 https://github.com/OpenTreeMap/otm-legacy/blob/v1.3/treemap/management/commands/dumpdata_iter.py

Here is an example of dumping OTM1 data out as fixtures

# user fixtures
python manage.py dumpdata_iter auth.user > /home/azavea/exports/latest/fixtures/users.json
python manage.py dumpdata_iter profiles.userprofile > /home/azavea/exports/latest/fixtures/userprofiles.json

# comment fixtures
python manage.py dumpdata_iter contenttypes > /home/azavea/exports/latest/fixtures/content_types.json
python manage.py dumpdata_iter threadedcomments.threadedcomment > /home/azavea/exports/latest/fixtures/threadedcomments.json

# main fixtures
python manage.py dumpdata_iter treemap.treephoto > /home/azavea/exports/latest/fixtures/treephotos.json
python manage.py dumpdata_iter treemap.species > /home/azavea/exports/latest/fixtures/species.json

# fixtures we might use for custom fields
python manage.py dumpdata_iter treemap.treealert > /home/azavea/exports/latest/fixtures/treealert.json
python manage.py dumpdata_iter treemap.treewatch > /home/azavea/exports/latest/fixtures/treewatch.json
python manage.py dumpdata_iter treemap.treefavorite > /home/azavea/exports/latest/fixtures/treefavorite.json

# slow fixtures
python manage.py dumpdata_iter treemap.treeresource > /home/azavea/exports/latest/fixtures/treeresource.json
python manage.py dumpdata_iter treemap.plot > /home/azavea/exports/latest/fixtures/plots.json
python manage.py dumpdata_iter treemap.tree > /home/azavea/exports/latest/fixtures/trees.json
python manage.py audit_export --outfile /home/azavea/exports/latest/fixtures/audits.json --errorfile /home/azavea/exports/latest/fixtures/audit_rejects.json

# boundaries
python manage.py dumpdata_iter treemap.neighborhood > /home/azavea/exports/latest/fixtures/neighborhoods.json

# reputation (so we can determine who should have the "Editors" role)
python manage.py dumpdata_iter django_reputation.reputation > /home/azavea/exports/latest/fixtures/reputation.json

The migrator was built more as an internal tool for moving data from OTM1 instances to opentreemap.org so unfortunately it is largely undocumented. It has been used successfully multiple times and was built to handle failures. You can restart a migration that failed in the middle and it will not create duplicate data. The management command that actually executes the migration is a good place to start exploring the system.

https://github.com/OpenTreeMap/otm-core/blob/dcbb0ff45e86388fb0d5d64e33e7df62d65acd15/opentreemap/otm1_migrator/management/commands/perform_migration.py

--
You received this message because you are subscribed to the Google Groups "opentreemap-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to opentreemap-user+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ulhas Shah]

Dear Justin,

Thanks for quick reply. We tried running perform_migrations but haven't got success. Following are details of process:

We are running OTM1 instance from long time (OTM1 code + data generated out of OTM1 code, http://trees.metastudio.org). Now we wanted to upgrade OTM1 --> OTM2 (OTM Core).

So tried following:

1. Created a new instance with OTM2 clone.

    1.1 Updated requirements.txt for [functools32==3.2.3.post2] to [functools32==3.2.3-2], as it was throwing an error of version not found.

2. Restored postgres data from OTM1 to OTM2.

3. Tried running  python manage.py perform_migration.

and now we are getting below error,

Invalid instance provided.tm/app/opentreemap# python manage.py perform_migration
Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 363, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 355, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 283, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 339, in execute
    self.stdout.write(output)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 107, in write
    if ending and not msg.endswith(ending):
AttributeError: 'int' object has no attribute 'endswith'

Can you guide us for migration steps? Where are we going wrong?

PS: Let us know if you need any additional details.

Thanks,

Ulhas Shah

To unsubscribe from this group and stop receiving emails from it, send an email to opentreemap-us...@googlegroups.com.

[Ulhas Shah]

Hi,

Please give any solution to this migration process.