crawl and push to solr index duplication errors

20 views
Skip to first unread message

Cinvoke

unread,
May 3, 2016, 5:43:45 AM5/3/16
to scrapy-users
hello !  im trying to pipeline my crawled data into solr but running into duplication issues.  i added a pipeline to keep track of ids so this wouldnt happen but it doesnt seem to be helping.
  
i keep getting the error: "Reason: Document is missing mandatory uniqueKey field: id".  
i think ive covered all the parts i can find.  im sure its something small ive left out. 
here is the full info: http://dpaste.com/29DBBR6

Thanks!

Dimitris Kouzis - Loukas

unread,
May 28, 2016, 2:54:56 PM5/28/16
to scrapy-users
Interesting. Did you solve this one?
Reply all
Reply to author
Forward
0 new messages