Yield request, but callback never called

[Leonardo Lazzaro]

Ok I think I know what the problem is, but I dont know how to solve it.

the thing is this, I have this code in a for (with 1000 objects to iterate)

for gallery in galleries:

   yield Request(

                                        url=self.upload_url,

                                        callback=self.upload_file,

                                        meta={ 'gallery' : gallery },

                                )

since self.upload_url is already "crawled" the request are never made.

I find using scrapy for POST is really difficult, seems more for crawling than to create "bots" (the good ones of course!)

thanks!

Leonardo

[Shane Evans]

You can use dont_filter=True on the Request object to make sure the
request is not filtered by the scheduler (see
http://doc.scrapy.org/topics/request-response.html#request-objects ).
This is just making the same request many times though, it may not be
what you intend.

To POST in scrapy you may want to use FormRequest, as documented here:
http://doc.scrapy.org/topics/request-response.html#using-formrequest-to-send-data-via-http-post

> --
> You received this message because you are subscribed to the Google
> Groups "scrapy-users" group.
> To post to this group, send email to scrapy...@googlegroups.com.
> To unsubscribe from this group, send email to
> scrapy-users...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/scrapy-users?hl=en.

[Leonardo Lazzaro]

Thanks dont_filter=True solved the problem.

I dont remember why I dont choose FormRequest, I am uploading files and I think I got some problems with FormRequest that time (around scrapy 0.10) not sure,  but the actual code works :) I also create some custom mime type to post it.
I will review to use formrequest (I use this for login now, but no for file upload)