Re: [google-appengine] Downloading only recent data from Datastore

[Sameer Lodha]

This would require Custom Logic to be implemented by you.

My suggestion...

1. Define an auto-update time-stamp property for each Entity Group

2. Write Logic to Download Data based on the time-stamp

3. If required, store the Last downloaded time-stamp value for each Entity Group, so that you may proceed from there in the next iteration.

The above is what we have implemented for incremental offline backups.

On Wed, Oct 17, 2012 at 12:15 AM, Mikhail Dubov <msd...@gmail.com> wrote:

Just read an article on how to download data from AppEngine Datastore:

https://developers.google.com/appengine/docs/python/tools/uploadingdata

It's a great tutorial, but, as far as I can see, it says nothing on whether it is possible to download entities from Datastore that were created after a specific date, which is actually my current need - I want to download my data periodically and each time I do this it would be preferrable to get the new/updated entities only. So, is that possible?

Thanks,

Mikhail

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/R4N9oJ-rOUkJ.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[alex]

BTW, there's a built-in '__version__' property of '__entity_group__'
that could be used instead of an auto-updated timestamp:

https://developers.google.com/appengine/docs/python/datastore/metadataentityclasses#EntityGroup
https://developers.google.com/appengine/docs/python/ndb/metadata

[Mikhail Dubov]

Thank you!

Is the post_export_function, as described in the article, the only way to implement Custom Logic? The problem is that, as far as I can see, the entities get filtered only after they are downloaded by the Bulkloader, that is, on the client side (that's why the function is called post export). This process is rather slow, though. Is there a possibility to perform that filtering on the server? Something like "pre_export_function"?

Thank you,

Mikhail

среда, 17 октября 2012 г., 10:24:12 UTC+4 пользователь Sam написал: