Problem with crawling ssl website

[Morad Edwar]

Hello,
I'm trying to crawl ssl website but i am stuck with this error in the spider and scrapy shell :

2015-01-05 11:35:04+0200 [ecsonlinestore2] ERROR: Error downloading <GET https://ecsonlinestore.com/>: [<twisted.python.failure.Failure <class 'OpenSSL.SSL.Error'>>]

I think it's because the "SNI Support" which is needed by the website, is there any plugin to solve this problem??
Scrapy Version : 0.20.2
Python Version : 2.7.6

[Nicolás Alejandro Ramírez Quiros]

This may help: https://github.com/scrapy/scrapy/issues/981

[Travis Leleu]

Morad -

It appears like the site you're scraping has an older configuration on their SSL that means the CA chain doesn't work as expected.  I'd look into how you can turn off SSL validation / strict checking so it doesn't try to validate the site identity.  After all, you probably don't care about the security component, you just want the data!

On Mon, Jan 5, 2015 at 6:49 AM, Nicolás Alejandro Ramírez Quiros <nrami...@gmail.com> wrote:

This may help: https://github.com/scrapy/scrapy/issues/981

--
You received this message because you are subscribed to the Google Groups "scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scrapy-users...@googlegroups.com.
To post to this group, send email to scrapy...@googlegroups.com.
Visit this group at http://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/d/optout.