New Bulkloader vs the deprecated Loader class

[Sylvain]

Hi,

I've tried to convert my "old" Loader class to the new Bulkloader.
I must admit, I've failed ;)

First, I've found that with the new Bulkloader, it's very easy to donwload datas.
Uploading, is another story... particularly if you want to execute post functions on entities.

Here is a list of thing that I didn't manage to do/understand,.... with the new bulkloader

• how to add external modules (sys.path) else all files must be in the same dir.
  
• how to execute a method attached to a model
• the new bulkloader doesn't check the model (required, type, default....)
• db.ComputedProperty or even DateTimeProperty with auto_add_ are not executed
• in the examples (http://bulkloadersample.appspot.com/showfile/bulkloader.yaml), it adds "import model", I thought it was "the key to make it works". But no.. I don't understand why I have to that, it changes nothing but it creates Kind with name "models.MyModel". Bad.
• the create_bulkloader_config function prepares the config_file for the download, to upload datas you have to add all transforms import else datas like (geo, list,...) are imported as string.
• post_import_function returns a dict not an entity. It's probably better for performance but more complicated
  

Most of these are easier with the Loader class.

You can find on Stackoverflow or in this group about : import_template, export, post_import_function, post_export_function but nothing really usefull in the official documentation (examples ?).

To make it works, you have to spend more time to prepare the datas than with the old Loader class.

Currenlty I use the loader class and the transform.py from the new Bulkloader.

Any other experiences with the new bulkloader ?

[Casey Dwyer]

I've tried it. I liked it. It's cleaner than the old loader. I ran into all the same problems as you as well as one with empty lists.

The import transform I used for lists was "lambda x: x.splitlines() or None" which worked well for importing them, but when I'd try to access an entity with empty list in my app I'd get a "that property is required" error even though the property was not required. My solution was to change the Model to an Expando and not declare the list fields.

Another caveat is that every row that can be indexed does get indexed whether you want it to or not. Eventually I just gave up with bulkloader and wrote a handler for writing a multiple entities to the datastore and a script for posting them to the handler. Took less effort than setting up bulkloader with none of the headaches, although I'm sure it's not nearly as efficient. :)