Copy entries from CSV file to Google App Engine Datastore

[Ludwig Precious]

Hi. I hope someone can help me with this. I have one CSV file with approximately 100 rows of data. Something like this:

Individuals.csv:

description,key,name,lastname
aaaaa aaaaa,1,Aname,Alastname
bbbbb bbbbb,2,Bname,Blastname
...
...

Inside my datastore I have an Entity Individual which I need to populate with that data from Individuals.csv.

I saw a couple of solutions but it doesn't work for me. Maybe they changed something how it's done. Any advice would be greatly appreciated. I am trying to do this whole day.

Thank you!

[Vinny P]

You can configure the App Engine bulkloader to automatically upload the file and create the relevant entities: https://developers.google.com/appengine/docs/python/tools/uploadingdata#Python_Configuring_the_bulk_loader . The only issue is that you'll have to create a YAML file to describe what the entity will look like (the process is outlined in the above link).

If you don't want to use the bulkloader, another option is to write an application to parse the CSV file and create the entities. Reading CSV files is fairly straightforward, there are libraries available for it within most languages. For example, here's the Go CSV parser: http://golang.org/pkg/encoding/csv/ and one for Python: https://docs.python.org/2/library/csv.html . This is a better route to go if you frequently have to upload new or updated CSV files.

 

 

-----------------

-Vinny P

Technology & Media Advisor

Chicago, IL

App Engine Code Samples: http://www.learntogoogleit.com

[husayt]

I am using Jackson for csv import in appengine and strongly recommend it.

https://github.com/FasterXML/jackson-dataformat-csv

[Ludwig Precious]

Thanks. I am a complete beginner in all this so I just wanted to ask you do you have an example of the application how it should look like? For example you mean like separate Android application which I will use only for CSV upload?

[Ludwig Precious]

Thank you for suggestion. Can you please show me some example how do you use it and connect to datastore? I just started learning and I am not sure how should I start.

[Vinny P]

On Thu, Apr 3, 2014 at 8:47 AM, Ludwig Precious <ludwig....@gmail.com> wrote:

I am a complete beginner in all this so I just wanted to ask you do you have an example of the application how it should look like? 

The correct answer depends on the programming language you're most comfortable in. For instance, if you're using Python there's example code here on how to upload a CSV file and parse it: http://stackoverflow.com/questions/2970599/upload-and-parse-csv-file-with-google-app-engine 

Getting that data into the datastore is the easy part: you can look at the guestbook example to see how entities are written to the datastore: https://github.com/GoogleCloudPlatform/appengine-guestbook-python