Data Migration from Development to Production Environment

[Sachindra Singh]

Hello all,

I have a development environment (running off a frequently updated git pull) on my laptop which has seen about 70~ data layers uploaded over a period of time. I want to migrate this data to a production environment - manually uploading all the layers would take more time than I can spare.

How would I go about moving the layers to a production environment and updating the psql db? Is there any scripts? I tried manually copying over gs-data folders from dev to prod machine but how do I reflect/migrate changes to psql?

Sachin

[David Winslow]

There are actually three or four stores of data to worry with in a GeoNode site:

• Django's database
  
• GeoNetwork's database
• GeoServer's data directory
• The PostGIS spatial store, if you're using (not active by default)
  

The Django frontend and GeoNetwork both default to using internal databases but can be configured to use PostgreSQL.  If you do things that way then migration is pretty easy: you can export your databases using Postgres's pg_dump utility (on your laptop/development environment) and import using psql or pg_restore (on your server/production environment). Something like this:

sachin@dev: $ pg_dump -d geonetwork -f geonetwork.sql && pg_dump -d geonode -f geonode.sql && scp geonode.sql geonetwork.sql sachin@prod:

sachin@prod: $ psql -d geonetwork -f geonetwork.sql && psql -d geonode -f geonode.sql

If you're using the default database for GeoNetwork instead, then it is stored within the exploded war directory (webapps/geonetwork/ in your working directory if you are using the Jetty configuration that's included with the sources.)  I would recommend simply transferring the entire web app, but ensure that GeoNetwork is not running when you do.  Alternatively, you can just install a clean GeoNetwork instance on the production server and use the django-admin.py updatelayers script to get your metadata into it.

If you are using the default database for Django, you can migrate to Postgres by using the loaddata and dumpdata commands.  These import and export the Django database in a database-independent format, suitable for both migrating between machines and migrating from one database to another.

sachin@dev: $ django-admin.py dumpdata --settings=geonode.settings > geonode.data.json && scp geonode.data.json sachin@prod:

sachin@prod: $ django-admin.py loaddata --settings=geonode.settings geonode.data.json

The GeoServer data should work when simply copied across.  If you need to change database connection parameters it's a bit trickier, especially if you've been using the experimental upload-to-postgis functionality which I believe creates a new datastore (and thereby another copy of the database connection parameters.) You could use GeoServer's REST API to correct it, although I think it would probably be more pragmatic to do some find-and-replace operations within $GEOSERVER_DATA_DIR/workspaces/*/datastore.xml .  However, when you do that it is quite easy to get GeoServer into a state where it won't let you edit the parameters through the web interface! So making a backup first is highly recommended.

Once all the data is migrated over, you'll probably run into trouble with the search page, where links in search results point to your development server.  You can fix it by making sure the SITEURL in local_settings.py is configured properly, then running the django-admin.py updatelayers command.

Hope this helps.

--
David Winslow
OpenGeo - http://opengeo.org/

[jadav...@gmail.com]

Hi David,

Your reply to previous question was quite helpful to me. But I now have altogether a different issue. I have data in GeoNode's geonode_data postgres database, but I removed the data directory from the GeoServer. Is there any way I can recreate the data directory from the postgres database?

Thanks,

Jadav