[Errno 8] _ssl.c:499: EOF occurred in violation of protocol

286 views
Skip to first unread message

Dave

unread,
Jan 20, 2012, 12:42:43 AM1/20/12
to gevent: coroutine-based Python network library
I am running 20 threads downloading various webpages through https
proxies and occasionally receive the following error:

Traceback (most recent call last):
File "/home/dave/projects/gks/env-gks/local/lib/python2.7/site-
packages/gevent/greenlet.py", line 390, in run
result = self._run(*self.args, **self.kwargs)
File "/home/dave/projects/gks/src/gksaccount.py", line 45, in scrape
raw_csv = self._csv_download(post_data, headers)
File "/home/dave/projects/gks/src/account.py", line 245, in
_csv_download
proxy=self.proxy)
File "/home/dave/projects/gks/env-gks/local/lib/python2.7/site-
packages/webscraping/download.py", line 128, in get
html = self.fetch(url, headers=headers, data=data, proxy=proxy,
user_agent=user_agent, opener=opener)
File "/home/dave/projects/gks/env-gks/local/lib/python2.7/site-
packages/webscraping/download.py", line 202, in fetch
response = opener.open(urllib2.Request(url, data, headers))
File "/usr/lib/python2.7/urllib2.py", line 391, in open
response = self._open(req, data)
File "/usr/lib/python2.7/urllib2.py", line 409, in _open
'_open', req)
File "/usr/lib/python2.7/urllib2.py", line 369, in _call_chain
result = func(*args)
File "/usr/lib/python2.7/urllib2.py", line 1193, in https_open
return self.do_open(httplib.HTTPSConnection, req)
File "/usr/lib/python2.7/urllib2.py", line 1160, in do_open
raise URLError(err)
URLError: <urlopen error [Errno 8] _ssl.c:499: EOF occurred in
violation of protocol>
<Greenlet at 0xb5689ac: <bound method GKSAccount.scrape of
<gksaccount.GKSAccount object at 0x932024c>>> failed with URLError

I did some Google searching a read about the 1024 file descriptor
limit and another library "leaking" sockets. It might be a red
herring, but I'm throwing it out there.

Thanks for the great library and help!
Dave
Reply all
Reply to author
Forward
0 new messages