bulkload_client.py not working

[Gampesh Sahu]

Hi,

I tried to upload abc.txt and abc.csv but it not uploaded it gives me
following error.


INFO 2008-11-17 13:16:41,967 bulkload_client.py] Starting import;
maximum 10 entities per post
INFO 2008-11-17 13:16:41,967 bulkload_client.py] Importing 4
entities in 29 bytes
ERROR 2008-11-17 13:16:43,437 bulkload_client.py] An error occurred
while importing: Received code 302: Found

302 Moved
The document has moved
here.
ERROR 2008-11-17 13:16:43,453 bulkload_client.py] Import failed

what is wrong in this.

[Marzia Niccolai]

Hi,

What information is in the logs for the posts from the client?  What does the App Engine handler look like for the data?  What does your app.yaml look like?

-Marzia

[Gampesh Sahu]

My app.yaml is

handlers:
- url: /load
script: loader.py
login: admin
- url: /.*
script: main.py

loader.py is

from google.appengine.ext import bulkload

class InventoryLoader(bulkload.Loader):
def __init__(self):
fields = [
("name", str),
("quantity", int)
]

bulkload.Loader.__init__(self, "Inventory", fields)

if __name__ == "__main__":
bulkload.main(InventoryLoader())

my inventory.csv is in the c:/ and i fire a command that is

C:\Program Files\google_appengine>tools\bulkload_client.py --
filename=c:\inventory.txt --king=Inventory --url=http://
famousandspicy.appspot.com/load

after getting error i fired another command which i got from the
google search

C:\Program Files\google_appengine>tools\bulkload_client.py --
filename=c:\inventory.txt --king=Inventory --url=http://
famousandspicy.appspot.com/load --cookie='ACSID=AJKiYc....[...]..w51g'

but i am not lucky this time also.


please help me out

Thanks
Gampesh

On Nov 18, 3:31 am, Marzia Niccolai <ma...@google.com> wrote:
> Hi,
>
> What information is in the logs for the posts from the client? What does
> the App Engine handler look like for the data? What does your app.yaml look
> like?
>
> -Marzia
>

[Marzia Niccolai]

Hi,

The 302 means that login is required, so you need to make sure you are passing in the correct --cookie parameter to the command line.

Visit famousandspicy.appspot.com/load in the browser to retrieve the cookie to use for the bulkuploader.

-Marzia

[Gampesh Sahu]

hi,
but i have passed --cookie which is generated after visiting "famousandspicy.appspot.com/load" but still it is not working.

thanks

[Marzia Niccolai]

Is the error still 302? Or did it generate another error? The 302 definitely indicates that the authentication isn't working, but a different error would mean something else.

Also, reading over the docstring for bulkload, it seems you can simplify this code by just having the file be:

inventory = bulkload.Loader('Inventory', [("name", str),("quantity", int)])

if __name__ == "__main__":
  bulkload.main(inventory)


-Marzia

[Gampesh Sahu]

yes i am getting same error that

302 Moved
  The document has moved
  here.
  ERROR 2008-11-17 13:16:43,453 bulkload_client.py] Import failed

[Marzia Niccolai]

Might be worth temporarily removing the admin restriction from the handler, omitting the cookie, and then trying to upload.  If this works, you know there is a problem with the cookie, so then you can sign out of your account, flush the browser cache, and try to re-generate a cookie for the site.

-Marzia

[Gampesh Sahu]

hi,

i did it in different browser but was not lucky i am very frustrate. i am burning my time since last two days, still burning.

thanks Marzia for your kind help.


Thanks

[Marzia Niccolai]

Does it work when you remove the login: admin from the app.yaml for this handler?

-Marzia

[Gampesh Sahu]

it shows error to me if i remove login:admin from app.yaml

[Gampesh Sahu]

Thanks a lot Marzia i really appreciate your troubleshooting you are
great :) thanks

Now data is importing comma separated CSV file but what if in my data
have "," as a content.


Thanks

On Nov 19, 12:14 am, Gampesh <gamp...@gmail.com> wrote:
> it shows error to me if i remove login:admin from app.yaml
>
> On Wed, Nov 19, 2008 at 12:42 AM, Marzia Niccolai <ma...@google.com> wrote:
> > Does it work when you remove the login: admin from the app.yaml for this
> > handler?
>
> > -Marzia
>

> > On Tue, Nov 18, 2008 at 10:53 AM, Gampesh <gamp...@gmail.com> wrote:
>
> >> hi,
>
> >> i did it in different browser but was not lucky i am very frustrate. i am
> >> burning my time since last two days, still burning.
>
> >> thanks Marzia for your kind help.
>
> >> Thanks
>
> >> On Wed, Nov 19, 2008 at 12:18 AM, Marzia Niccolai <ma...@google.com>wrote:
>
> >>> Might be worth temporarily removing the admin restriction from the
> >>> handler, omitting the cookie, and then trying to upload. If this works, you
> >>> know there is a problem with the cookie, so then you can sign out of your
> >>> account, flush the browser cache, and try to re-generate a cookie for the
> >>> site.
>
> >>> -Marzia
>

> >>> On Tue, Nov 18, 2008 at 10:34 AM, Gampesh <gamp...@gmail.com> wrote:
>
> >>>> yes i am getting same error that
>
> >>>> 302 Moved
> >>>> The document has moved
> >>>> here.
> >>>> ERROR 2008-11-17 13:16:43,453 bulkload_client.py] Import failed
>
> >>>> On Tue, Nov 18, 2008 at 11:45 PM, Marzia Niccolai <ma...@google.com>wrote:
>
> >>>>> Is the error still 302? Or did it generate another error? The 302
> >>>>> definitely indicates that the authentication isn't working, but a different
> >>>>> error would mean something else.
>
> >>>>> Also, reading over the docstring for bulkload, it seems you can
> >>>>> simplify this code by just having the file be:
>
> >>>>> inventory = bulkload.Loader('Inventory', [("name", str),("quantity",
> >>>>> int)])
>
> >>>>> if __name__ == "__main__":
> >>>>> bulkload.main(inventory)
>
> >>>>> -Marzia
>

> >>>>> On Tue, Nov 18, 2008 at 9:50 AM, Gampesh <gamp...@gmail.com> wrote:
>
> >>>>>> hi,
> >>>>>> but i have passed --cookie which is generated after visiting "
> >>>>>> famousandspicy.appspot.com/load" but still it is not working.
>
> >>>>>> thanks
>
> >>>>>> On Tue, Nov 18, 2008 at 11:01 PM, Marzia Niccolai <ma...@google.com>wrote:
>
> >>>>>>> Hi,
>
> >>>>>>> The 302 means that login is required, so you need to make sure you
> >>>>>>> are passing in the correct --cookie parameter to the command line.
>
> >>>>>>> Visit famousandspicy.appspot.com/load in the browser to retrieve the
> >>>>>>> cookie to use for the bulkuploader.
>
> >>>>>>> -Marzia
>

> >>>>>>>> famousandspicy.appspot.com/load--cookie='ACSID=AJKiYc....[...]..w51g'

[Sargis Dallakyan]

I had the same problem with bulk load; removing 'login: admin' from
app.yaml solved the problem. Its strange that this works on Linux and
fails on Windows. This might have something to do with copy/paste on
cmd window.

Regarding CSV data format, csv module in python lets you choose
delimiter http://www.python.org/doc/2.5.2/lib/csv-examples.html and it
would be nice if this can be passed as a command line option for
bulkload_client.py. I was able to modify my local App Engine SDK to
have '|' as a delimiter, for instance, but when it was time to upload
data to appspot.com, I replaced '.' with ':' in my data with a simple
python script [1].

Also bulkload_client.py can;t handle Unicode so you'll need to replace
them with ASCII. You can convert them back to Unicode with Data Viewer
later on if needed.

[1] Here is the script that I used to convert ',' to ':'. Note that my
input file has delimiter='|'.

import sys, csv
fileName = sys.argv[1]
reader = csv.reader(open(fileName, "rb"), delimiter='|',)
out = open('mod_'+fileName,'w')
for row in reader:
for item in row[:-1]:
out.write(item.replace(',',':'))
out.write(',')
out.write(row[-1].replace(',',':'))
out.write('\n')
out.close()