migrating database and updating it

[Yebach]

Hello

So based on all the material I have read about migration etc some of the updates are not working. I will explain what I am doing and please correct me where my steps are wrong

I going from develop to production version and the key is to keep the database with its current data. 

1. I backup my production database
   
2. I created new directory with my develop web2py files - DAL is different in a way i have some new tables and some new fields compare to my production version of the database
   
3. copied all the files and folders to create a new app
4. deleted ALL files in App/database folder
5. restored database on Postgres under new name
6. Set migrate = True and fake_migrate = True in 0.py file 
7. run web2py with web2py.exe -i 192.168.100.47 -p 8000 -a 12345 -K WoShiTest -X

Now the tables are not updated and also new tables are not added.

What am I doing wrong?

I am using the version of web2py  downloaded on 11.10.2017

Thank you

Best regards

[Anthony]

Are you trying to keep the production database and just run migrations on it, or just create a whole new production database from scratch to match the new code? If the former, don't touch the /databases folder on production -- just update the code and leave migrations on, and the DAL will run the proper migrations to make the production database match the new models.

Anthony

[Vid Ogris]

I am trying to keep my production db with all its data but update the structure with the new web2py db code

--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to a topic in the Google Groups "web2py-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/web2py/opmKbIW-7LA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to web2py+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Anthony]

On Wednesday, October 18, 2017 at 10:37:17 AM UTC-4, Yebach wrote:

I am trying to keep my production db with all its data but update the structure with the new web2py db code

Yes, then as noted below, just update the web2py code, make sure migrations are on, and do not touch the production /databases folder (the *.table files there provide a record of the current state of the production database, so web2py knows what migrations to run to make the database match the new model code).

Anthony

[Vid Ogris]

Hello

So as said I copied the whole app directory to my new server, also restored my database on postgres.  

First my settings for migrate and false_migrate are set to False and immediately I get an error: relation of table auth_user already exists. I set the migrate for this table (auth.define_tables(username=False, signature=False, migrate = False) ) to False, then I get an error relation of table scheduler already exists. So i set migrate = False for that table also in my scheduler.py file. And afterwards the app itself works. 

I already find this kind of behavior strange since I copied "stuff" and things don't work --> I would appreciate guidance for further migrations already here. I need to keep my auth_user table data etc.

Now I want to use my new db.py file and if I set migrate to True - I get an error immediately that table xyz already exists. 

Any suggestions 

--

Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to a topic in the Google Groups "web2py-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/web2py/opmKbIW-7LA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to web2py+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Lep pozdrav 

Vid Ogris

[Anthony]

It's not quite clear exactly what you have copied and/or changed, so hard to say what you need to do. But in general, if you want to update your code and have the production database migrated to match the new code, you should copy everything except the contents of the /databases directory (leave the production /databases directory untouched). Then after the code has been copied, make sure migrations are enabled, and on the first request, the production database will be migrated to match the new models. Sounds like you didn't follow that process and things have gotten out of sync.

Anthony

On Monday, October 23, 2017 at 6:54:22 AM UTC-4, Yebach wrote:

Hello

So as said I copied the whole app directory to my new server, also restored my database on postgres.  

First my settings for migrate and false_migrate are set to False and immediately I get an error: relation of table auth_user already exists. I set the migrate for this table (auth.define_tables(username=False, signature=False, migrate = False) ) to False, then I get an error relation of table scheduler already exists. So i set migrate = False for that table also in my scheduler.py file. And afterwards the app itself works. 

I already find this kind of behavior strange since I copied "stuff" and things don't work --> I would appreciate guidance for further migrations already here. I need to keep my auth_user table data etc.

Now I want to use my new db.py file and if I set migrate to True - I get an error immediately that table xyz already exists. 

Any suggestions 

[Alex Glaros]

I'm a beginner but seems like you should create your new version to first be exact duplicate of the original, pg_restore db to the new version, make sure it's working like old one, THEN change DAL tables, then run migrate = True.  (I haven't needed to run fake_migrate in these situations)

Maybe I read it wrong but looks like you changed DAL first on the new instance, then imported Postgres db.

Of course keep your Postgres archive as permanent backup.

Alex Glaros

[Yebach]

So this is the whole procedure I did at the end and it seems to do the trick

1. create a folder for your new app (WoShiTest)
2. copy production version there
3. delete content of sessions, error and DATABASE folders
4. in 0.py set db connection string
5. set migrate = True and false migrate = False
6. create database in postgres with the name you put in 0.py
7. start web2py (in my case from cmd running web2py.exe -i 192.168.100.47 -p 8080 -a 12345 -K WoShiTest -X)
8. check if db is created and access to the webapp
9. it works? Good
10. delete database on postgres
11. create new with the same name and restore the production db with all your data
12. no errors or warnings? Good
13. again run web2py server and check if all the stuff works - now including login stuff
14. it does? No! You are getting ERROR:web2py.scheduler.ue-ebi#2488:    error popping tasks --> stop server. delete all scheduler related tables on postgres and files in database folder related to scheduler table
15. run server again
16. now it works? Great - almost there
    
17. now lets update the current app to the new/develop app
18. copy the following folders from develop app - controlers, model, modules, static, ...view and all you think you need EXCEPT errors, sessions and of course DATABASE
19. update 0.py and set new db connection
20. now start using your brand new updated bugfree app :)

I hope somebody finds this useful