Long XMLs are not being indexed in Solr 4.8

[George Karakatsiotis]

While I am indexing (with Solr 4.8) long xml files (more than 32K) I am getting the following exception:

Failed to index document with PubId: 40121897 in SearchServer!
Cause: Exception writing document id 40121897 to the index; possible analysis error.
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Exception writing document id 40121897 to the index; possible analysis error.
at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:554)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
at org.apache.solr.client.solrj.impl.LBHttpSolrServer.doRequest(LBHttpSolrServer.java:340)
at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:301)
at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:636)
at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:124)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)

 

According to https://issues.apache.org/jira/browse/LUCENE-5472, if a term is longer than 32K (in our case the lux_xml is longer than that) an exception is being thrown, and the document isn't being added in the index.

[George Karakatsiotis]

It seems that if you change the lux_xml field to be a stored field but not an indexed one, you are able to index/retrieve the xml document successfully.