Migrate a PostgreSQL Webfaction DB to Opalstack

[Gaël Princivalle]

Hi.

I'm still trying to migrate my Web2py applications from Webfaction to Opalstack.

Well here what I do:

1 I export the DB form Webfaction in the dump.sql file

What are the best options?

For one website only the Data's in the Copy mode has worked well.

For another one I've loose some records.

Any suggestion?

2 I create a new PostgreSQL DB in Opalstack.

3 I modify my DB string connection in Web2py like that:

db = DAL('postgres://mydbuser:mydbpassword@localhost:5432/mydb', check_reserved=['all'], pool_size=1, entity_quoting=True, bigint_id=True, migrate=True, fake_migrate_all=False)

The idea is to create the tables before importing the data.

So here there is the problem to understand if setting migrate and fake:migrate_all to True or False. Any suggestion?

4 I import the data in the new DB:

psql -U mydbuser mydb < dump.sql

Depending of the attempts I have these errors:

relation "table..." does not exist

Some tables are not imported (empty)

Web2py don't find the images stored in records

Any idea is interesting, thanks.

[António Ramos]

i had the same path.

Images you should , i think, just download from webfaction and upload to opalstack to uploads folder.

I used dbeaver to export from webfaction . and then psql to insert in opalstack.

I had the same problem but i used migrate=False. Check docs. 

Not much but hope it helps. 

Ramos

--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups "web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/web2py/5dac3729-f70c-405b-81c8-a0c7fa0112d7n%40googlegroups.com.

[Jose C]

Opalstack have some migration tools available: https://community.opalstack.com/d/155-howto-migrate-stuff-automatically-from-webfaction

Also I found my old notes from when I moved my web2py apps and PG databases over 18 months ago, hopefully they will help.  Also includes how to sync images directories between servers...

1) sync photos directories between WF and OS.   On OS run:

rsync -avh --dry-run   <username>@web565.webfaction.com:/home/<wf_username>/virtenvs/<appname>/lib/web2py/applications/<appname>/static/photos    /home/<os_username>/virtenvs/<appname>/lib/web2py/applications/<appname>/static  > results_photos.txt

logs what would happen to the results_photos.txt file.

When ready to execute, remove the --dry-run parameter above.

2) on WF server:
    pg_dump -U <db_user> -d <db_name> -h localhost -p <portnum> -f re_livedump.sql
    gzip re_livedump.sql
   
   
3) copy to OS server and gunzip the file on the server.

4) ensure web2py server is not running (remember to comment out crontab autostart line as well)
delete contents of the web2py/applications/<appname>/databases directory.

5) on OS server, go into psql and remove existing tables, etc. if they exist:
    psql -U <db_user> -d <db_name>
    DROP OWNED BY <db_user>;
   
6) load new database
    psql -U <db_user> -d <db_name> -f re_livedump.sql
   
there may be a few error messages about the owner being different, safe to ignore.

7) set web2py DAL flags:
    migrate=True
    fake_migrate=True
    lazy_tables=False

8) start up web2py.  Verify all table files created in the databases directory.

9) shutdown web2py and set the above flags to their inverses.  Also uncomment the crontab autostart line.

10) startup again and verify site is working as expected.  An option is to mount your new OS web2py app on a temp domain, sub-domain or even on <username>.opalstacked.com for testing.

HTH,

[António Ramos]

rsync is awesome :) 

thank you

Em ter., 27 de out. de 2020 às 13:00, Jose C <houdini...@gmail.com> escreveu:

Opalstack have some migration tools available: https://community.opalstack.com/d/155-howto-migrate-stuff-automatically-from-webfaction

Also I found my old notes from when I moved my web2py apps and PG databases over 18 months ago, hopefully they will help.  Also includes how to sync images directories between servers...

1) sync photos directories between WF and OS:

rsync -avh --dry-run   <username>@web565.webfaction.com:/home/<wf_username>/virtenvs/<appname>/lib/web2py/applications/<appname>/static/photos    /home/<os_username>/virtenvs/<appname>/lib/web2py/applications/<appname>/static  > results_photos.txt

logs what would happen to the results_photos.txt file.

When ready to execute, remove the --dry-run parameter above.

2) on WF server:
    pg_dump -U <db_user> -d <db_name> -h localhost -p <portnum> -f re_livedump.sql
    gzip re_livedump.sql
   
   
3) copy to OS server and gunzip the file on the server.

4) ensure web2py server is not running (remember to comment out crontab autostart line as well)
delete contents of the web2py/applications/<appname>/databases directory.

5) on OS server, go into psql and remove existing tables, etc. if they exist:
    psql -U <db_user> -d <db_name>
    DROP OWNED BY <db_user>;
   
6) load new database
    psql -U <db_user> -d <db_name> -f re_livedump.sql
   
there may be a few error messages about the owner being different, safe to ignore.

7) set web2py DAL flags:
    migrate=True
    fake_migrate=True
    lazy_tables=False

8) start up web2py.  Verify all table files created in the databases directory.

9) shutdown web2py and set the above flags to their inverses.  Also uncomment the crontab autostart line.

10) startup again and verify site is working as expected.  An option is to mount your new OS web2py app on a temp domain, sub-domain or even on <username>.opalstacked.com for testing.

HTH,

--

Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups "web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/web2py/a7031961-424b-46c7-9c3a-4b1a02ca46f1o%40googlegroups.com.

[Jose C]

On Tuesday, 27 October 2020 at 14:35:23 UTC Ramos wrote:

rsync is awesome :) 

thank you

Yes, a very nifty little utility... and if you backup to any cloud providers, then check out rclone as well (built on rsync).