After second run getting scrapy.exceptions.NotSupported: Unsupported URL scheme 'doublehttp': no handler available for that scheme exception

[Matthias Brück]

Hey,

I run into an exception after a second crawl run, when items should get updated. Somehow 'doublehttp' is prepended to the detail page url:

2016-10-19 23:42:24 [scrapy] ERROR: Error downloading <GET doublehttp://xxx.de/event/1794>

Traceback (most recent call last):

  File "/Users/brx/.environments/notifhain/lib/python3.4/site-packages/twisted/internet/defer.py", line 1183, in _inlineCallbacks

    result = result.throwExceptionIntoGenerator(g)

  File "/Users/brx/.environments/notifhain/lib/python3.4/site-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator

    return g.throw(self.type, self.value, self.tb)

  File "/Users/brx/.environments/notifhain/lib/python3.4/site-packages/scrapy/core/downloader/middleware.py", line 43, in process_request

    defer.returnValue((yield download_func(request=request,spider=spider)))

  File "/Users/brx/.environments/notifhain/lib/python3.4/site-packages/scrapy/utils/defer.py", line 45, in mustbe_deferred

    result = f(*args, **kw)

  File "/Users/brx/.environments/notifhain/lib/python3.4/site-packages/scrapy/core/downloader/handlers/__init__.py", line 64, in download_request

    (scheme, self._notconfigured[scheme]))

scrapy.exceptions.NotSupported: Unsupported URL scheme 'doublehttp': no handler available for that scheme

The first run is fine and the items get stored in the database. But the second somehow gets messed up. I just use a pre_url processer and tried to make everything as explained in the get-started.

Has anyone an idea?

Thank you!

Best,

Matthias