errors when running portia spiders

69 views
Skip to first unread message

Henry John

unread,
Apr 13, 2017, 8:20:47 AM4/13/17
to portia-scraper
When click the “Run this spider” icon in webui, the error message:
2017-04-13 20:16:46+0800 [_GenericHTTPChannelProtocol,2,127.0.0.1] Unhandled Error
        Traceback (most recent call last):
          File "/usr/local/lib/python2.7/dist-packages/twisted/web/http.py", line 1767, in allContentReceived
            req.requestReceived(command, path, version)
          File "/usr/local/lib/python2.7/dist-packages/twisted/web/http.py", line 768, in requestReceived
            self.process()
          File "/usr/local/lib/python2.7/dist-packages/twisted/web/server.py", line 183, in process
            self.render(resrc)
          File "/usr/local/lib/python2.7/dist-packages/twisted/web/server.py", line 234, in render
            body = resrc.render(self)
        --- <exception caught here> ---
          File "/usr/local/lib/python2.7/dist-packages/scrapyd/webservice.py", line 21, in render
            return JsonResource.render(self, txrequest).encode('utf-8')
          File "/usr/local/lib/python2.7/dist-packages/scrapyd/utils.py", line 20, in render
            r = resource.Resource.render(self, txrequest)
          File "/usr/local/lib/python2.7/dist-packages/twisted/web/resource.py", line 250, in render
            return m(request)
          File "/usr/local/lib/python2.7/dist-packages/scrapyd/webservice.py", line 49, in render_POST
            spiders = get_spider_list(project, version=version)
          File "/usr/local/lib/python2.7/dist-packages/scrapyd/utils.py", line 137, in get_spider_list
            raise RuntimeError(msg.encode('unicode_escape') if six.PY2 else msg)
        exceptions.RuntimeError: Scrapy 1.2.2 - no active project\n\nUnknown command: list\n\nUse "scrapy" to see available commands\n


and When run with command:  portiacrawl data/projects/HaitunCun "www.haituncun.com"
the error is
2017-04-13 20:19:39 [scrapy] DEBUG: Telnet console listening on 127.0.0.1:6023
DEBUG:scrapy.core.engine:Crawled (200) <GET http://www.haituncun.com/ymuyin/> (referer: None)
2017-04-13 20:19:39 [scrapy] DEBUG: Crawled (200) <GET http://www.haituncun.com/ymuyin/> (referer: None)
WARNING:py.warnings:/usr/local/lib/python2.7/dist-packages/scrapy/link.py:21: UserWarning: Link urls must be str objects. Assuming utf-8 encoding (which could be wrong)
  warnings.warn("Link urls must be str objects. "

2017-04-13 20:19:40 [py.warnings] WARNING: /usr/local/lib/python2.7/dist-packages/scrapy/link.py:21: UserWarning: Link urls must be str objects. Assuming utf-8 encoding (which could be wrong)
  warnings.warn("Link urls must be str objects. "

INFO:scrapy.core.engine:Closing spider (finished)

What is wrong?
Reply all
Reply to author
Forward
0 new messages