DSPace solr error

64 views

Skip to first unread message

Andres Mateo Gomez Gallego

unread,

May 21, 2020, 5:54:30 PM5/21/20

to DSpace Technical Support

After restarting Tomcat, our DSpace stopped listing collections.

Also when browsing by date, the following error is shown:

http://repositorio.utp.edu.co/dspace/handle/11059/47/browse?type=dateissued

Expected mime type application/octet-stream but got text/html. <html><head><title>Apache Tomcat/7.0.67 - Informe de Error</title><style></style> </head><body><h1>Estado HTTP 500 - {msg=SolrCore 'search' is not available due to init failure: Error opening new searcher,trace=org.apache.solr.common.SolrException: SolrCore 'search' is not available due to init failure: Error opening new searcher at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:745) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:299) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.dspace.solr.filters.LocalHostRestrictionFilter.doFilter(LocalHostRestrictionFilter.java:50) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:423) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1079) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.solr.common.SolrException: Error opening new searcher at org.apache.solr.core.SolrCore.<init>(SolrCore.java:873) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:646) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:491) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:255) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:249) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622) ... 1 more Caused by: org.apache.solr.common.SolrException: Error opening new searcher at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1565) at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1677) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:845) ... 8 more Caused by: java.io.EOFException: read past EOF: MMapIndexInput(path="/dspace/dspace_i/solr/search/data/index/segments_ehh") at org.apache.lucene.store.ByteBufferIndexInput.readByte(ByteBufferIndexInput.java:81) at org.apache.lucene.store.BufferedChecksumIndexInput.readByte(BufferedChecksumIndexInput.java:41) at org.apache.lucene.store.DataInput.readInt(DataInput.java:98) at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:343) at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:454) at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:906) at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:752) at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:450) at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:792) at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:77) at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:64) at org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:279) at org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:111) at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1528) ... 10 more ,code=500}</h1><HR size="1" noshade="noshade"><p><b>type</b> Informe de estado</p><p><b>mensaje</b> <u>{msg=SolrCore 'search' is not available due to init failure: Error opening new searcher,trace=org.apache.solr.common.SolrException: SolrCore 'search' is not available due to init failure: Error opening new searcher at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:745) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:299) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.dspace.solr.filters.LocalHostRestrictionFilter.doFilter(LocalHostRestrictionFilter.java:50) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:423) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1079) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.solr.common.SolrException: Error opening new searcher at org.apache.solr.core.SolrCore.<init>(SolrCore.java:873) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:646) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:491) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:255) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:249) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622) ... 1 more Caused by: org.apache.solr.common.SolrException: Error opening new searcher at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1565) at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1677) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:845) ... 8 more Caused by: java.io.EOFException: read past EOF: MMapIndexInput(path="/dspace/dspace_i/solr/search/data/index/segments_ehh") at org.apache.lucene.store.ByteBufferIndexInput.readByte(ByteBufferIndexInput.java:81) at org.apache.lucene.store.BufferedChecksumIndexInput.readByte(BufferedChecksumIndexInput.java:41) at org.apache.lucene.store.DataInput.readInt(DataInput.java:98) at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:343) at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:454) at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:906) at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:752) at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:450) at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:792) at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:77) at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:64) at org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:279) at org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:111) at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1528) ... 10 more ,code=500}</u></p><p><b>descripción</b> <u>El servidor encontró un error interno que hizo que no pudiera rellenar este requerimiento.</u></p><HR size="1" noshade="noshade"><h3>Apache Tomcat/7.0.67</h3></body></html>

El contenido de este mensaje y sus anexos son únicamente para el uso del destinatario y pueden contener información clasificada o reservada. Si usted no es el destinatario intencional, absténgase de cualquier uso, difusión, distribución o copia de esta comunicación.

Jose Blanco

unread,

May 21, 2020, 9:31:02 PM5/21/20

to Andres Mateo Gomez Gallego, DSpace Technical Support

I'm guessing your solr webapp is not up. Did you deploy it with tomcat? Look at my tomcat /solr webapp deployed in tomcat in the image I've attached.

-Jose

--
All messages to this mailing list should adhere to the DuraSpace Code of Conduct: https://duraspace.org/about/policies/code-of-conduct/
---
You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dspace-tech/be475234-d7d0-4e93-be55-449ceebfe2a1%40googlegroups.com.

Screen Shot 2020-05-21 at 9.29.36 PM.png

Mark H. Wood

unread,

May 22, 2020, 11:37:39 AM5/22/20

to DSpace Technical Support

On Thu, May 21, 2020 at 02:45:43PM -0700, Andres Mateo Gomez Gallego wrote:
> After restarting Tomcat, our DSpace stopped listing collections.
>
> Also when browsing by date, the following error is shown:
>
> http://repositorio.utp.edu.co/dspace/handle/11059/47/browse?type=dateissued
>
>
> Expected mime type application/octet-stream but got text/html.
> <html><head><title>Apache Tomcat/7.0.67 - Informe de
> Error</title><style></style>
> </head><body><h1>Estado HTTP 500 - {msg=SolrCore 'search' is not available
> due to init failure: Error opening new
> searcher,trace=org.apache.solr.common.SolrException: SolrCore 'search' is
> not available due to init failure: Error opening new searcher at

... much removed ...

> Caused by: java.io.EOFException: read past EOF:
> MMapIndexInput(path="/dspace/dspace_i/solr/search/data/index/segments_ehh")
> at
> org.apache.lucene.store.ByteBufferIndexInput.readByte(ByteBufferIndexInput.java:81)

I've never seen this happen, but it appears that the 'search' core's
file that holds the current list of index segments is corrupt. The
simplest thing to do may be to delete all of the core's data files
(every file in solr/search/data/) and re-index with something like
'bin/dspace index-discovery -b -s'. If you have a lot of items in
your repository, this could take a long time.

Further reading:

https://lucene.apache.org/core/3_0_3/fileformats.html#Segments%20File
https://support.lucidworks.com/s/question/0D58000003KjCBMCA3/how-to-recover-if-the-solr-index-got-corrupted
https://lucidworks.com/post/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/

I don't think that recovery without re-indexing is possible without a
good segments_N file, but maybe someone with more Solr expertise could
comment?

--
Mark H. Wood
Lead Technology Analyst

University Library
Indiana University - Purdue University Indianapolis
755 W. Michigan Street
Indianapolis, IN 46202
317-274-0749
www.ulib.iupui.edu