Error when using scrapy shell

[Mario]

Hi, so I have a issue when trying to inspect elements at this page

Here’s what I enter in terminal:

scrapy shell "https://hrsa.cunyfirst.cuny.edu/psc/cnyhcprd/GUEST/HRMS/c/COMMUNITY_ACCESS.CLASS_SEARCH.GBL"

And this is the output that I get:

exceptions.AttributeError: 'NoneType' object has no attribute 'failVerification'

Here’s full output from terminal: http://pastebin.com/eEhmaLr5

Only solution that I found online was that I need to install service_identity which I already have. Any help is appreciated.

​

[Mario]

A little update, can somebody just test out running shell with above mentioned url? So I can see if its error just on my end.

[Travis Leleu]

Just tried it w/ scrapy 1.0 and it loaded the shell and page fine.  Just trying to launch the scrapy shell gives you that error?

--
You received this message because you are subscribed to the Google Groups "scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scrapy-users...@googlegroups.com.
To post to this group, send email to scrapy...@googlegroups.com.
Visit this group at http://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/d/optout.

[Mario]

Yes. You can see output on pastebin above.

[Travis Leleu]

Looks like the exception happens when trying to validate the security certificate of the web server.  Try updating your version of twisted to the latest.  If that doesn't work, please provide summary information of your OS, scrapy version, python version, twisted version.  And a separate list of your install python packages, including versions.

Also, your email address bounces back, mailbox rejected by hotmail.  Please fix this by updating your email address in the google group preferences.

[Mario]

I updated twisted version from 15.3.0 to 15.4.0.

Here’s information about OS:

Linux asd-PC 3.16.0-51-generic #69~14.04.1-Ubuntu SMP Wed Oct 7 15:32:41 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

Scrapy version is: Scrapy 1.0.3

Python version is: Python 2.7.6

Here’s output from running command pip freeze: http://pastebin.com/dwrj73gv

I updated the email thing.

[Travis Leleu]

After updating twisted, you receive the same error?  I'm afraid I don't have any further suggestions other than checking that all of your libraries are up to date.

[Mario]

I just fixed the issue, I have run command: sudo pip install scrapy.

Here's the output: http://pastebin.com/h8pZXZJj

So it seems like it was issue with service_identity after all.

Thanks for the help Travis, I really appreciate it.

[Alejandro Mazzuca]

Hi, It worked for me . http://prntscr.com/8u5d0b
I could open it.

[bruce]

Hi Mario.

Are you trying to scrape the cunyfirst college class schedule site?

What are you trying to really get/accomplish. Let me know, might be
able to help you out.