SolrException - multiple values encountered for non multiValued field (binary files as components)
Gary Stutton
I have encountered SEVERE errors in the logs when modelling binary files as components.
To check and recreate the error I did the following:
- I created a new site (using 'empty' blueprint)
- I created a content type 'component' with one datasource (file browse from repo) and one linked item selector
- I edited a 'page' content type to add one data source (child content) and one linked item selector
- I then edited a sample page and added two new components - both with a single file (which in my test case were both Freemarker text files, FTL, but on our proof of concept site can be any binary files)
This produces the following error in the log (extract):
Nov 27, 2013 3:08:58 PM org.apache.solr.common.SolrException log
SEVERE: org.apache.solr.common.SolrException: ERROR: [doc=test-gps:/site/website/index.xml] multiple values encountered for non multiValued field attachments.item.component.attachment.item.fileType_s: [FTL, FTL]
at org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:246)
at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:60)
at org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:115)
at org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:157)
at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:79)
at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:58)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1376)
at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:365)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:260)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:470)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:615)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:602)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:662)
Nov 27, 2013 3:08:58 PM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr-crafter path=/update params={wt=xml&version=2.2} status=400 QTime=0
ERROR: org.craftercms.search.controller.SearchRestController - RESTful request /crafter-search/api/1/search/update failed
org.craftercms.search.exception.SearchException: Server at http://localhost:8080/solr-crafter returned non ok status:400, message:Bad Request
at org.craftercms.search.service.impl.SolrSearchService.update(SolrSearchService.java:135)
at org.craftercms.search.controller.SearchRestController.update(SearchRestController.java:86)
at sun.reflect.GeneratedMethodAccessor537.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:213)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:126)
***
Note - in our proof of concept site I am also getting the same error on a different field orderDefault_f
I have looked into the schema.xml (through the solr admin gui) and found this section:
<dynamicField name="*_i" type="int" indexed="true" stored="true"/>
<dynamicField name="*_s" type="string" indexed="true" stored="true"/>
<dynamicField name="*_l" type="long" indexed="true" stored="true"/>
<dynamicField name="*_t" type="text_general" indexed="true" stored="true"/>
<dynamicField name="*_txt" type="text_general" indexed="true" stored="true" multiValued="true"/>
<dynamicField name="*_en" type="text_en" indexed="true" stored="true" multiValued="true"/>
<dynamicField name="*_b" type="boolean" indexed="true" stored="true"/>
<dynamicField name="*_f" type="float" indexed="true" stored="true"/>
<dynamicField name="*_d" type="double" indexed="true" stored="true"/>
It appears to me that file uploads create metadata with fileSize_s and pages with placeInNav add metadata for orderDefault_f and these are not treated as multivalued in the solr schema that ships with Crafter, which means adding more than one to a page creates this error.
I want to model binary attachments as components so that if the linked attachment changes we can change the component to point to the new file without having to find all the pages that reference that file (and I don't think Crafter provides any 'link management' to find those pages anyway).
In the case of the orderDefault_f issue I am modelling certain items as 'pages' and then also using them elsewhere in the site as 'child content items' so a single page might contain content from several 'child' pages. Hence the error message for multiple values of orderDefault_f presumably from the 'include' statements on the child content.
I am guessing the problem can be fixed by setting the dynamicFields above to be multiValued="true" for "*_s" and "*_f", but would this cause other issues within Crafter? Or is there another solution to this problem?
Many thanks,
Gary