Discovery is broken

297 views

Skip to first unread message

kpet

unread,

Apr 9, 2021, 6:34:18 AM4/9/21

to DSpace Technical Support

I have a Dspace 6.3 installed on a linux machine, but searching/browsing the repository, is not working (error stacktrace provided below). Everything else is fine. The discovery interface is non existent in the xmlui front page (see the attached photo), which leads me to believe it's a configuration issue, but I cannot pinpoint it.

I have ensured that the user tomcat runs as, has write permissions on the [dspace] directory, and I have edited the tomcat9.service file to include these 2 lines under the [Service] section:

ReadWritePermissions=[dspace]/assetstore
ReadWritePermissions=[dspace]/webapps/solr

In addition, I have verified that the solr indexer is indeed located at the value of {solr.server} in both local.cfg and dspace.cfg via performing:

wget -O - http://localhost:8080/solr/search/select?q=test

as stated in the docs, however, I get a 403 (Forbidden) response form that.

The xmlui error stacktrace is the following:

Expected mime type application/octet-stream but got text/html. <!doctype html><html lang="en"><head><title>HTTP Status 500 – Internal Server Error</title><style type="text/css">body {font-family:Tahoma,Arial,sans-serif;} h1, h2, h3, b {color:white;background-color:#525D76;} h1 {font-size:22px;} h2 {font-size:16px;} h3 {font-size:14px;} p {font-size:12px;} a {color:black;} .line {height:1px;background-color:#525D76;border:none;}</style></head><body><h1>HTTP Status 500 – Internal Server Error</h1><hr class="line" /><p><b>Type</b> Status Report</p><p><b>Message</b> {msg=SolrCore 'search' is not available due to init failure: Error opening new searcher,trace=org.apache.solr.common.SolrException: SolrCore 'search' is not available due to init failure: Error opening new searcher at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:745) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:299) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.dspace.solr.filters.LocalHostRestrictionFilter.doFilter(LocalHostRestrictionFilter.java:50) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:688) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1639) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.solr.common.SolrException: Error opening new searcher at org.apache.solr.core.SolrCore.<init>(SolrCore.java:881) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:654) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:491) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:255) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:249) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ... 1 more Caused by: org.apache.solr.common.SolrException: Error opening new searcher at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1574) at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1686) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:853) ... 8 more Caused by: java.nio.file.FileSystemException: /dspace/solr/search/data/index/write.lock: Read-only file system at sun.nio.fs.UnixException.translateToIOException(UnixException.java:91) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:177) at java.nio.channels.FileChannel.open(FileChannel.java:287) at java.nio.channels.FileChannel.open(FileChannel.java:335) at org.apache.lucene.store.NativeFSLock.obtain(NativeFSLockFactory.java:167) at org.apache.lucene.store.Lock.obtain(Lock.java:77) at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:755) at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:77) at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:64) at org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:284) at org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:116) at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1537) ... 10 more ,code=500}</p><p><b>Description</b> The server encountered an unexpected condition that prevented it from fulfilling the request.</p><hr class="line" /><h3>Apache Tomcat/9.0.31 (Ubuntu)</h3></body></html>

error.PNG

Mark H. Wood

unread,

Apr 9, 2021, 8:38:31 AM4/9/21

to dspac...@googlegroups.com

On Fri, Apr 09, 2021 at 03:34:18AM -0700, kpet wrote:
> I have a Dspace 6.3 installed on a linux machine, but searching/browsing
> the repository, is not working (error stacktrace provided below).
> Everything else is fine. The discovery interface is non existent in the
> xmlui front page (see the attached photo), which leads me to believe it's a
> configuration issue, but I cannot pinpoint it.

It is actually a volume mounting issue. Your Solr 'search' core is on
a volume which for some reason was mounted read-only. Solr is unable
to open its lock file for writing, and so cannot load the core.

> I have ensured that the user tomcat runs as, has write permissions on the
> [dspace] directory, and I have edited the tomcat9.service file to include
> these 2 lines under the [Service] section:
> ReadWritePermissions=[dspace]/assetstore
> ReadWritePermissions=[dspace]/webapps/solr
>
> In addition, I have verified that the solr indexer is indeed located at the
> value of {solr.server} in both local.cfg and dspace.cfg via performing:
>
> wget -O - http://localhost:8080/solr/search/select?q=test
>
> as stated in the docs, however, I get a 403 (Forbidden) response form that.
>
> The xmlui error stacktrace is the following:
>
> Expected mime type application/octet-stream but got text/html. <!doctype

> html><html lang="en"><head><title>HTTP Status 500 ??? Internal Server

> Error</title><style type="text/css">body
> {font-family:Tahoma,Arial,sans-serif;} h1, h2, h3, b
> {color:white;background-color:#525D76;} h1 {font-size:22px;} h2
> {font-size:16px;} h3 {font-size:14px;} p {font-size:12px;} a {color:black;}
> .line
> {height:1px;background-color:#525D76;border:none;}</style></head><body><h1>HTTP

> Status 500 ??? Internal Server Error</h1><hr class="line" /><p><b>Type</b>

> Status Report</p><p><b>Message</b> {msg=SolrCore 'search' is not
> available due to init failure: Error opening new

This is typical of a problem with starting or contacting Solr.

^^^^^^^^^^^^^^^^^^^^^ Here is the problem.
/dspace/solr/search/data/index/write.lock is not writable.

> sun.nio.fs.UnixException.translateToIOException(UnixException.java:91) at
> sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at
> sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at
> sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:177)
> at java.nio.channels.FileChannel.open(FileChannel.java:287) at
> java.nio.channels.FileChannel.open(FileChannel.java:335) at
> org.apache.lucene.store.NativeFSLock.obtain(NativeFSLockFactory.java:167)
> at org.apache.lucene.store.Lock.obtain(Lock.java:77) at
> org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:755) at
> org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:77)
> at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:64)
> at
> org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:284)
> at
> org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:116)
> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1537) ... 10
> more ,code=500}</p><p><b>Description</b> The server encountered an
> unexpected condition that prevented it from fulfilling the request.</p><hr
> class="line" /><h3>Apache Tomcat/9.0.31 (Ubuntu)</h3></body></html>

--
Mark H. Wood
Lead Technology Analyst

University Library
Indiana University - Purdue University Indianapolis
755 W. Michigan Street
Indianapolis, IN 46202
317-274-0749
www.ulib.iupui.edu