Data Backup and Restore

[Prashant Hegde]

Hello !

I am trying to figure out how to backup and restore data in appscale.

https://github.com/AppScale/appscale/wiki/Migration-into-AppScale,-Backup-&-Recovery

The above link seems outdated.

For example,

"python $APPSCALE_HOME/AppDB/backup/scripts/backup_data.py --app-id APP_ID --source-code"

The path of the backup_data.py is incorrect.

Even if you run appscale/datastore/scripts/backup_data.py, we get this error:

"ValueError: Attempted relative import in non-package"

So,I changed it to :

"python -m datastore.scripts.backup_data -a localapp" ( ran from directory appscale)

The script just exits - it doesnt seem to do much. 

Is there a updated doc somewhere?  Or can you please let me know what is the right way to backup the data on appscale?

Regards
Prashant

 

[Chris Donati]

That command was moved to a console entry point. You can run `appscale-backup-data -a localapp`, and it should write the backup to /opt/appscale/backups/. The corresponding restore command is `appscale-restore-data`.

I will update the wiki with the new commands. Just as a heads up, I've run into an issue before where appscale-restore-data did not write the composite indexes for the restored entities, so I had to run appscale-update-index to populate them.

--
You received this message because you are subscribed to the Google Groups "AppScale Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to appscale_community+unsub...@googlegroups.com.
To post to this group, send email to appscale_community@googlegroups.com.
Visit this group at https://groups.google.com/group/appscale_community.
For more options, visit https://groups.google.com/d/optout.

[Prashant Hegde]

Thanks Chris, that worked.

I will keep your advise in mind when we restore!

On Fri, Dec 15, 2017 at 11:34 AM, Chris Donati <chris....@appscale.com> wrote:

That command was moved to a console entry point. You can run `appscale-backup-data -a localapp`, and it should write the backup to /opt/appscale/backups/. The corresponding restore command is `appscale-restore-data`.

I will update the wiki with the new commands. Just as a heads up, I've run into an issue before where appscale-restore-data did not write the composite indexes for the restored entities, so I had to run appscale-update-index to populate them.

On Thu, Dec 14, 2017 at 9:56 PM, Prashant Hegde <prashan...@gmail.com> wrote:

Hello !

I am trying to figure out how to backup and restore data in appscale.

https://github.com/AppScale/appscale/wiki/Migration-into-AppScale,-Backup-&-Recovery

The above link seems outdated.

For example,

"python $APPSCALE_HOME/AppDB/backup/scripts/backup_data.py --app-id APP_ID --source-code"

The path of the backup_data.py is incorrect.

Even if you run appscale/datastore/scripts/backup_data.py, we get this error:

"ValueError: Attempted relative import in non-package"

So,I changed it to :

"python -m datastore.scripts.backup_data -a localapp" ( ran from directory appscale)

The script just exits - it doesnt seem to do much. 

Is there a updated doc somewhere?  Or can you please let me know what is the right way to backup the data on appscale?

Regards
Prashant

 

--
You received this message because you are subscribed to the Google Groups "AppScale Community" group.

To unsubscribe from this group and stop receiving emails from it, send an email to appscale_community+unsubscribe@googlegroups.com.

To post to this group, send email to appscale_community@googlegroups.com.
Visit this group at https://groups.google.com/group/appscale_community.
For more options, visit https://groups.google.com/d/optout.

[Prashant Hegde]

Hi Chris,

I was able to successfully backup the data. But, when I try to restore, it encountered an issue. Seems like there is a bug in the script.

 When I executed this:

appscale-restore-data -a localapp --backup-dir /opt/appscale/backups/localapp-20171215-142241/ -c -d

I got an error:

"Seems that "localapp" is not deployed."

The script seems to be checking /var/apps directory to see if the app is deployed. However, the directory name check is not matching.

When an app is deployed, the directory name is "localapp_default_v1_1513328444342"

However, the script expects the directory name to be "localapp"

When I renamed the directory to "localapp" the script went through fine.

Is this expected?

Prashant

[Prashant Hegde]

Also:

appscale-update-index -a localapp -t cassandra

When I try to update indexes, it asks a serial number of the index to update. We have 131 indexes. Is there no way to update all indexes?

Thanks in advance.

[Chris Donati]

The data restore issue you mentioned is definitely a bug. When we implemented services, we changed the application directory structure. I'll look into fixing that bug.

I've submitted a pull request for the second issue that you mentioned here: https://github.com/AppScale/appscale/pull/2631. It allows you to update all of the indexes without selecting them one by one.

To unsubscribe from this group and stop receiving emails from it, send an email to appscale_community+unsub...@googlegroups.com.

[Prashant Hegde]

Thanks Chris. Appreciate you looking into it.. 

Prashant