Scrapy 1.1 RC4 is out!

[Paul Tremberth]

Hello, Scrapy users!

We are happy to announce Scrapy 1.1 Release Candidate 4!

What's new in 1.1.0rc4 compared to 1.1.0rc3:

- HTTPS connections work much better, and for more real world cases ;

  thank you for all your feedback on these issues.

- canonicalize_url() output changes a bit, especially for non-ASCII URLs;

  this is technically backward incompatible (although we consider it as a bug fix)

  so double check your link extractors (which use it by default)

  and HTTP caches (as it is used by the default request fingerprinting)

+ a handful of bug fixes, new features and documentation improvements.

See all the changes between 1.1.0rc3 and 1.1.0rc4 at this URL:

https://github.com/scrapy/scrapy/compare/1.1.0rc3...1.1.0rc4

or check the release notes changes (easier to digest):

https://github.com/scrapy/scrapy/pull/1960/files

Scrapy 1.1 RC4 is available on PyPI, as universal wheel and source distribution.

Please give it a try, and report any problems at: https://github.com/scrapy/scrapy/issues

Install Scrapy 1.1.0rc4 via pip (we recommend that you use a virtualenv):

    $ pip install scrapy==1.1.0rc4

e.g., in a Python3 virtualenv:

    $ mkvirtualenv --python=/usr/bin/python3 scrapy11rc4.py3
    (scrapy11rc4.py3)$ pip install scrapy==1.1.0rc4

Final note: make sure you upgrade w3lib to the latest 1.14.2 version which came out a few weeks ago,

you'll need it to get proper behavior for non-ASCII URLs.

Thank you and happy scraping!

/Paul, for the Scrapy team.

[Paul Tremberth]

Hi Scrapy users!

Have you tried Scrapy 1.1 RC4 yet (see instructions in previous message below)?

If not, please do to help us find final bugs before the final 1.1 release.

If you did try it and it worked well, that's also valuable feedback, so let us know :-)

Happy scraping!

/Paul, for the Scrapy team.