Question about migrating to HRD: What happens if I revert and try to remigrate without deleting target app data?

[John Wheeler]

Can someone, preferably from the app engine team, or who has experience with this answer the following question:

I began migrating an M/S datastore app to HRD. It was taking forever during the 'Copy' phase, so I assumed it was stuck, and I reverted it. It is now stuck reverting, and I've filed a ticket for that, but that's a separate issue. 

I missed this part of the documentation, which states: 

"If you want to re-migrate to that target application at a later date, you must delete the target application's Datastore data."

What happens if I don't delete the data? It so happens that much of my data copied over - tens of millions of rows, which would cost hundreds if not thousands of dollars to delete. I am reluctant to just create a new app and migrate to it because I already got the mail quota removed from the current target, and that took a while.

What happens if you migrate and don't delete the data? Does it overwrite it? The migration tool gives you the option to perform the migration when data exists, but it gives you a warning and does specify what will happen if you don't delete the data. The docs clearly say you 'must' delete the target data, so I'm wondering technically what the ramifications are.

If you can answer this question without speculating, I'd be grateful. A successful migration without any hiccups and 100% data fidelity is very important to me.

Thanks, John

[Ryan Huebsch]

Hi John,

If you do not delete the data that already exists in target app, your target app may wind up with "extra" data that has since been deleted in the source app. Consider the following scenario:

- source app has entity, E1

- migration is started, E1 is copied to target app

- migration is reverted

- source app deletes E1

- migration is restarted, E1 may persist in the target app even though it was removed from the source app

This is the primary reason we suggest you clear the target app's datastore before beginning a migration, to ensure the target app's datastore has the same entities.

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.

[John Wheeler]

Ryan,

This is truly helpful. Thank you. It just so happens that my source app doesn't do any deletes. However, it does do updates. Is it safe to assume that when I restart the migration, my 'old' target entities will be replaced with my new source entities?

Thanks

John

--
You received this message because you are subscribed to a topic in the Google Groups "Google App Engine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/ktFeqM7kEws/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-appengi...@googlegroups.com.

[Ryan Huebsch]

Correct, when the migration is restarted from the start all existing entities in the source app will be re-copied, replacing whatever version of the entity may already exists.

Note, this is another reason we give the warning when transferring to a target app that already has data, in that any changes you've made in the target app may be lost. This particular caveat does not appear applicable to your situation.