Scrapy 1.1.0rc1 release candidate is out
48 views
Skip to first unread message
Elias Dorneles
Feb 4, 2016, 8:19:36 AM2/4/16
to scrapy-users
Hello, Scrapy users!
We are happy to announce that we just shipped Scrapy 1.1 release candidate 1, which brings beta Python 3 support to Scrapy!
The team has been working hard to bring this highly anticipated feature and we are very grateful to all developers and users who contributed to it -- thank you! =)
This version also comes with a lot of new features and bug fixes. Here is a quick summary highlighting the most relevant ones:
Hot new features:
Item loaders now support nested loaders.
FormRequest.from_response improvements.
Added setting AUTOTHROTTLE_TARGET_CONCURRENCY and improved AutoThrottle docs.
response.text to get response body as unicode.
Anonymous S3 connections are now supported.
Downloader middlewares now can return deferreds.
HTTP caching now follows RFC2616 more closely.
Selectors were extracted to the parsel library. This means you can use Scrapy Selectors without Scrapy and also upgrade the selectors engine without needing to upgrade Scrapy.
Behavior changes and fixes that may require your attention:
Scrapy doesn't retry bad requests (HTTP 400) by default. If you need the old behavior, add 400 to RETRY_HTTP_CODES.
Fix shell files argument handling. If you try scrapy shell index.html it will try to load the URL http://index.html, use scrapy shell ./index.html to load a local file.
Robots.txt compliance is now enabled by default for newly-created projects. Scrapy will also wait for robots.txt to be downloaded before proceeding with the crawl. If you want to disable this behavior, update ROBOTSTXT_OBEY in settings.py file after creating a new project.
Exporters now work on unicode, instead of bytes by default. If you use PythonItemExporter, you may want to update your code to disable binary mode which is now deprecated.
Accept XML node names containing dots as valid.
Check out other new features and fixes in the full release notes at: http://doc.scrapy.org/en/stable/news.html#id1
Please give Scrapy 1.1.0rc1 a try, check out the cool new features and Python 3 support, feel free to report any problems at: https://github.com/scrapy/scrapy/issues
Install Scrapy 1.1.0rc1 via pip:
$ pip install Scrapy==1.1.0rc1
We are happy to announce that we just shipped Scrapy 1.1 release candidate 1, which brings beta Python 3 support to Scrapy!
The team has been working hard to bring this highly anticipated feature and we are very grateful to all developers and users who contributed to it -- thank you! =)
This version also comes with a lot of new features and bug fixes. Here is a quick summary highlighting the most relevant ones:
Hot new features:
Item loaders now support nested loaders.
FormRequest.from_response improvements.
Added setting AUTOTHROTTLE_TARGET_CONCURRENCY and improved AutoThrottle docs.
response.text to get response body as unicode.
Anonymous S3 connections are now supported.
Downloader middlewares now can return deferreds.
HTTP caching now follows RFC2616 more closely.
Selectors were extracted to the parsel library. This means you can use Scrapy Selectors without Scrapy and also upgrade the selectors engine without needing to upgrade Scrapy.
Behavior changes and fixes that may require your attention:
Scrapy doesn't retry bad requests (HTTP 400) by default. If you need the old behavior, add 400 to RETRY_HTTP_CODES.
Fix shell files argument handling. If you try scrapy shell index.html it will try to load the URL http://index.html, use scrapy shell ./index.html to load a local file.
Robots.txt compliance is now enabled by default for newly-created projects. Scrapy will also wait for robots.txt to be downloaded before proceeding with the crawl. If you want to disable this behavior, update ROBOTSTXT_OBEY in settings.py file after creating a new project.
Exporters now work on unicode, instead of bytes by default. If you use PythonItemExporter, you may want to update your code to disable binary mode which is now deprecated.
Accept XML node names containing dots as valid.
Check out other new features and fixes in the full release notes at: http://doc.scrapy.org/en/stable/news.html#id1
Please give Scrapy 1.1.0rc1 a try, check out the cool new features and Python 3 support, feel free to report any problems at: https://github.com/scrapy/scrapy/issues
Install Scrapy 1.1.0rc1 via pip:
$ pip install Scrapy==1.1.0rc1
Thank you, happy scraping!
Uncharted
Feb 5, 2016, 6:10:28 AM2/5/16
to scrapy-users
Nice ! Good job guys :)
Reply all
Reply to author
Forward
0 new messages