I use scrapy 1.3 According to documentation I can declare settings for each specific spider simple enough. Here is my code
class BhSpider(scrapy.Spider):
name = "code"
custom_settings = {'CONCURRENT_REQUESTS' : '20',
'FEED_EXPORT_FIELDS' :['price', 'stock','partnumber','sku', 'name' ,'manufacture','attribute','distributor','upc','descr', 'p_url','main_image','images']}
allowed_domains = ***
But according to logs project settings are used. How do i know project settings are overcomed by this specific spider settings?
here is the log
2017-08-20 13:47:51 [scrapy.utils.log] INFO: Scrapy 1.3.2 started (bot: pc)
2017-08-20 13:47:51 [scrapy.utils.log] INFO: Overridden settings: {'NEWSPIDER_MODULE': 'pc.spiders'
, 'FEED_URI': 'test_output1.csv', 'CONCURRENT_REQUESTS': 200, 'SPIDER_MODULES': ['pc.spiders'], 'BO
T_NAME': 'pc', 'USER_AGENT': 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) C
hrome/59.0.3071.115 Safari/537.36', 'FEED_FORMAT': 'csv', 'FEED_EXPORT_FIELDS': ['price', 'stock',
'partnumber', 'sku', 'name', 'manufacture', 'attribute', 'distributor', 'upc', 'descr']}
2017-08-20 13:47:51 [scrapy.middleware] INFO: Enabled extensions:
In the output file Export fields seem to be correct( as in custom settings, though log does not show that). How can i know that specific settings are enabled?
