[Dspace-tech] Solr is not working with DSpace 4.1 returns HTTP Status 403-..

494 views

Skip to first unread message

Hassan Bhuiyan

unread,

Aug 26, 2015, 1:16:53 PM8/26/15

to dspac...@lists.sourceforge.net, Teo Tokis, Hilton Gibson

Hi all,

Many thanks to Toe and Hilton. I have tried as per your advice but unfortunately they didn’t work.

Now I am trying to check whether the fresh DSapce 4.1 works or not.

If it works then I will do the upgrade from DSpace 1.6.2.

I formatted the machine and reinstalled DSpace 4.1 with all prerequisite latest version software on SUSE Linux Enterprise Server 11 (x86_64).

Testing http://localhost:8080/jspui or http://localhost:8080/xmlui or http://localhost:8080/oai/request?verb=Identify works fine.

While testing http://localhost:8080/solr/ does not work though ../apache-tomcat-8.0.5/webapps/solr directory is copied.

It shows following error as below on browser. Any help would be highly appreciated.

Error or WARN messages on Browser

-------------------------------------------------------------------------------------------

HTTP Status 403-

Type Status report

Message

Description Access to the specified resources has been forbidden.

Apache Tomcat/8.0.5

Error or WARN message in dspace.log. 2014-06-19 file

---------------------------------------------------------------------------------------------

*** No ERROR or WARN in dspace.log. 2014-06-19 file except INFO

Error or WARN message in ../apache-tomcat-8.0.5/logs> vi catalina.out

------------------------------------------------------------------------------------------------

763 [coreLoadExecutor-3-thread-1] WARN org.apache.solr.core.SolrResourceLoader â Can't find (or read) directory to add to classloader: ../../../contrib/extraction/lib (resolved as: /dspace/dspace/solr/search/../../../contrib/extraction/lib).

770 [coreLoadExecutor-3-thread-1] WARN org.apache.solr.core.SolrResourceLoader â Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /dspace/dspace/solr/search/../../../dist).

770 [coreLoadExecutor-3-thread-1] WARN org.apache.solr.core.SolrResourceLoader â Can't find (or read) directory to add to classloader: ../../../contrib/clustering/lib/ (resolved as: /dspace/dspace/solr/search/../../../contrib/clustering/lib).

770 [coreLoadExecutor-3-thread-1] WARN org.apache.solr.core.SolrResourceLoader â Can't find (or read) directory to add to classloader: ../../../contrib/langid/lib/ (resolved as: /dspace/dspace/solr/search/../../../contrib/langid/lib).

771 [coreLoadExecutor-3-thread-1] WARN org.apache.solr.core.SolrResourceLoader â Can't find (or read) directory to add to classloader: ../../../contrib/velocity/lib (resolved as: /dspace/dspace/solr/search/../../../contrib/velocity/lib).

771 [coreLoadExecutor-3-thread-1] WARN org.apache.solr.core.SolrResourceLoader â Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /dspace/dspace/solr/search/../../../dist).

767 [coreLoadExecutor-3-thread-2] INFO org.apache.solr.core.SolrConfig â Adding specified lib dirs to ClassLoader

777 [coreLoadExecutor-3-thread-2] WARN org.apache.solr.core.SolrResourceLoader â Can't find (or read) directory to add to classloader: ../../../contrib/extraction/lib (resolved as: /dspace/dspace/solr/statistics/../../../contrib/extraction/lib).

778 [coreLoadExecutor-3-thread-2] WARN org.apache.solr.core.SolrResourceLoader â Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /dspace/dspace/solr/statistics/../../../dist).

778 [coreLoadExecutor-3-thread-2] WARN org.apache.solr.core.SolrResourceLoader â Can't find (or read) directory to add to classloader: ../../../contrib/clustering/lib/ (resolved as: /dspace/dspace/solr/statistics/../../../contrib/clustering/lib).

778 [coreLoadExecutor-3-thread-2] WARN org.apache.solr.core.SolrResourceLoader â Can't find (or read) directory to add to classloader: ../../../contrib/langid/lib/ (resolved as: /dspace/dspace/solr/statistics/../../../contrib/langid/lib).

778 [coreLoadExecutor-3-thread-2] WARN org.apache.solr.core.SolrResourceLoader â Can't find (or read) directory to add to classloader: ../../../contrib/velocity/lib (resolved as: /dspace/dspace/solr/statistics/../../../contrib/velocity/lib).

779 [coreLoadExecutor-3-thread-2] WARN org.apache.solr.core.SolrResourceLoader â Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /dspace/dspace/solr/statistics/../../../dist).

796 [coreLoadExecutor-3-thread-3] INFO org.apache.solr.core.SolrConfig â Adding specified lib dirs to ClassLoader

796 [coreLoadExecutor-3-thread-3] WARN org.apache.solr.core.SolrResourceLoader â Can't find (or read) directory to add to classloader: ../../../contrib/extraction/lib (resolved as: /dspace/dspace/solr/oai/../../../contrib/extraction/lib).

796 [coreLoadExecutor-3-thread-3] WARN org.apache.solr.core.SolrResourceLoader â Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /dspace/dspace/solr/oai/../../../dist).

796 [coreLoadExecutor-3-thread-3] WARN org.apache.solr.core.SolrResourceLoader â Can't find (or read) directory to add to classloader: ../../../contrib/clustering/lib/ (resolved as: /dspace/dspace/solr/oai/../../../contrib/clustering/lib).

796 [coreLoadExecutor-3-thread-3] WARN org.apache.solr.core.SolrResourceLoader â Can't find (or read) directory to add to classloader: ../../../contrib/langid/lib/ (resolved as: /dspace/dspace/solr/oai/../../../contrib/langid/lib).

796 [coreLoadExecutor-3-thread-3] WARN org.apache.solr.core.SolrResourceLoader â Can't find (or read) directory to add to classloader: ../../../contrib/velocity/lib (resolved as: /dspace/dspace/solr/oai/../../../contrib/velocity/lib).

2173 [coreLoadExecutor-3-thread-3] WARN org.apache.solr.core.RequestHandlers â Multiple requestHandler registered to the same name: /spell ignoring: org.apache.solr.handler.component.SearchHandler

2175 [coreLoadExecutor-3-thread-1] WARN org.apache.solr.core.RequestHandlers â Multiple requestHandler registered to the same name: /spell ignoring: org.apache.solr.handler.component.SearchHandler

2179 [coreLoadExecutor-3-thread-2] WARN org.apache.solr.core.RequestHandlers â Multiple requestHandler registered to the same name: /spell ignoring: org.apache.solr.handler.component.SearchHandler

19-Jun-2014 10:24:31.465 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory /usr/local/tomcat/apache-tomcat-8.0.5/webapps/sword

log4j:WARN No appenders could be found for logger (org.dspace.servicemanager.DSpaceKernelInit).

log4j:WARN Please initialize the log4j system properly.

log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

Error or WARN shows in ../apache-tomcat-8.0.5/logs>.. /dspace/log> vi solr.log

----------------------------------------------------------------------------------------------------

2014-06-19 10:24:29,631 WARN org.apache.solr.core.SolrResourceLoader @ Can't find (or read) directory to add to classloader: ../../../contrib/extraction/lib (resolved as: /dspace/dspace/solr/search/../../../contrib/extraction/lib).

2014-06-19 10:24:29,638 WARN org.apache.solr.core.SolrResourceLoader @ Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /dspace/dspace/solr/search/../../../dist).

2014-06-19 10:24:29,638 WARN org.apache.solr.core.SolrResourceLoader @ Can't find (or read) directory to add to classloader: ../../../contrib/clustering/lib/ (resolved as: /dspace/dspace/solr/search/../../../contrib/clustering/lib).

2014-06-19 10:24:29,638 WARN org.apache.solr.core.SolrResourceLoader @ Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /dspace/dspace/solr/search/../../../dist).

2014-06-19 10:24:29,638 WARN org.apache.solr.core.SolrResourceLoader @ Can't find (or read) directory to add to classloader: ../../../contrib/langid/lib/ (resolved as: /dspace/dspace/solr/search/../../../contrib/langid/lib).

2014-06-19 10:24:29,638 WARN org.apache.solr.core.SolrResourceLoader @ Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /dspace/dspace/solr/search/../../../dist).

2014-06-19 10:24:29,639 WARN org.apache.solr.core.SolrResourceLoader @ Can't find (or read) directory to add to classloader: ../../../contrib/velocity/lib (resolved as: /dspace/dspace/solr/search/../../../contrib/velocity/lib).

2014-06-19 10:24:29,639 WARN org.apache.solr.core.SolrResourceLoader @ Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /dspace/dspace/solr/search/../../../dist).

2014-06-19 10:24:29,635 INFO org.apache.solr.core.SolrConfig @ Adding specified lib dirs to ClassLoader

2014-06-19 10:24:29,645 WARN org.apache.solr.core.SolrResourceLoader @ Can't find (or read) directory to add to classloader: ../../../contrib/extraction/lib (resolved as: /dspace/dspace/solr/statistics/../../../contrib/extraction/lib).

2014-06-19 10:24:29,646 WARN org.apache.solr.core.SolrResourceLoader @ Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /dspace/dspace/solr/statistics/../../../dist).

2014-06-19 10:24:29,646 WARN org.apache.solr.core.SolrResourceLoader @ Can't find (or read) directory to add to classloader: ../../../contrib/clustering/lib/ (resolved as: /dspace/dspace/solr/statistics/../../../contrib/clustering/lib).

2014-06-19 10:24:29,646 WARN org.apache.solr.core.SolrResourceLoader @ Can't find (or read) directory to add to classloader: ../../../contrib/langid/lib/ (resolved as: /dspace/dspace/solr/statistics/../../../contrib/langid/lib).

2014-06-19 10:24:29,646 WARN org.apache.solr.core.SolrResourceLoader @ Can't find (or read) directory to add to classloader: ../../../contrib/velocity/lib (resolved as: /dspace/dspace/solr/statistics/../../../contrib/velocity/lib).

2014-06-19 10:24:29,647 WARN org.apache.solr.core.SolrResourceLoader @ Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /dspace/dspace/solr/statistics/../../../dist).

2014-06-19 10:24:29,664 INFO org.apache.solr.core.SolrConfig @ Adding specified lib dirs to ClassLoader

2014-06-19 10:24:29,664 WARN org.apache.solr.core.SolrResourceLoader @ Can't find (or read) directory to add to classloader: ../../../contrib/extraction/lib (resolved as: /dspace/dspace/solr/oai/../../../contrib/extraction/lib).

2014-06-19 10:24:29,664 WARN org.apache.solr.core.SolrResourceLoader @ Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /dspace/dspace/solr/oai/../../../dist).

2014-06-19 10:24:29,664 WARN org.apache.solr.core.SolrResourceLoader @ Can't find (or read) directory to add to classloader: ../../../contrib/clustering/lib/ (resolved as: /dspace/dspace/solr/oai/../../../contrib/clustering/lib).

2014-06-19 10:24:29,664 WARN org.apache.solr.core.SolrResourceLoader @ Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /dspace/dspace/solr/oai/../../../dist).

2014-06-19 10:24:29,664 WARN org.apache.solr.core.SolrResourceLoader @ Can't find (or read) directory to add to classloader: ../../../contrib/langid/lib/ (resolved as: /dspace/dspace/solr/oai/../../../contrib/langid/lib).

2014-06-19 10:24:29,664 WARN org.apache.solr.core.SolrResourceLoader @ Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /dspace/dspace/solr/oai/../../../dist).

2014-06-19 10:24:29,664 WARN org.apache.solr.core.SolrResourceLoader @ Can't find (or read) directory to add to classloader: ../../../contrib/velocity/lib (resolved as: /dspace/dspace/solr/oai/../../../contrib/velocity/lib).

2014-06-19 10:24:29,664 WARN org.apache.solr.core.SolrResourceLoader @ Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /dspace/dspace/solr/oai/../../../dist).

2014-06-19 10:24:31,041 WARN org.apache.solr.core.RequestHandlers @ Multiple requestHandler registered to the same name: /spell ignoring: org.apache.solr.handler.component.SearchHandler

2014-06-19 10:24:31,043 WARN org.apache.solr.core.RequestHandlers @ Multiple requestHandler registered to the same name: /spell ignoring: org.apache.solr.handler.component.SearchHandler

2014-06-19 10:24:31,047 WARN org.apache.solr.core.RequestHandlers @ Multiple requestHandler registered to the same name: /spell ignoring: org.apache.solr.handler.component.SearchHandler

CONTENT of ../dspace/config/modules> vi solr-statistics.cfg is

------------------------------------------------------------------------------

server = http://localhost:8080/solr/statistics

# A comma-separated list that contains the bundles for which the bitstreams will be displayed

query.filter.bundles=ORIGINAL

# control solr statistics querying to filter out spider IPs

# false by default

query.filter.spiderIp = false # Uncommented by me

# control solr statistics querying to look at "isBot" field to determine

# if record is a bot. true by default.

query.filter.isBot = true # Uncommented by me

# URLs to download IP addresses of search engine spiders from

spiderips.urls = http://iplists.com/google.txt, \

http://iplists.com/inktomi.txt, \

http://iplists.com/lycos.txt, \

http://iplists.com/infoseek.txt, \

http://iplists.com/altavista.txt, \

http://iplists.com/excite.txt, \

http://iplists.com/misc.txt, \

http://iplists.com/non_engines.txt

CONTENT of my ../dspace/config/modules> vi usage-statistics.cfg

------------------------------------------------------------------------------

# The location for the Geo Database retrieved on update/installation

dbfile = /dspace/dspace/config/GeoLiteCity.dat

# Timeout for the resolver in the DNS lookup

# Time in milliseconds, defaults to 200 for backward compatibility

# Your system's default is usually set in /etc/resolv.conf and varies

# between 2 to 5 seconds, too high a value might result in Solr exhausting

# your connection pool

resolver.timeout = 200

# Control if the statistics pages should be only shown to authorized users

# If enabled, only the administrators for the DSpaceObject will be able to

# view the statistics.

# If disabled, anyone with READ permissions on the DSpaceObject will be able

# to view the statistics.

#View/download statistics

authorization.admin.usage=true

#Search/search result statistics

authorization.admin.search=true

#Workflow result statistics

authorization.admin.workflow=true

# Enable/disable logging of spiders in solr statistics.

# If false, and IP matches an address in spiderips.urls, event is not logged.

# If true, event will be logged with the 'isBot' field set to true

# (see query.filter.* for query filter options)

# Default value is true.

logBots = true

Many thanks

Hassan B.

Brunel University, UK

From: Teo Tokis [mailto:teot...@gmail.com]
Sent: 13 June 2014 09:12
To: Hilton Gibson
Cc: Hassan Bhuiyan; dspac...@lists.sourceforge.net
Subject: Re: [Dspace-tech] Exception: Server at http://localhost:8080/solr/search returned non ok status:404, message:Not Found

Hello,

it seems like a Solr's problem.

A solution to your problem is to copy the folder "[dspace-installation-directory]\webapps\solr" to "C:\Program Files\Apache Software Foundation\Apache Tomcat (your version)\webapps" (or wherever is the Tomcat's installation directory).

Another solution is to locate the Tomcat's server.xml file and change the

with something like that

Then you can test if Solr is "working" @ http://localhost:8081/solr/

Hope it helps.

Teo.

2014-06-13 10:24 GMT+03:00 Hilton Gibson <hilton...@gmail.com>:

Hi Hassan

Perhaps this will help: http://wiki.lib.sun.ac.za/index.php/SUNScholar/SOLR_Statistics

Cheers

Hilton Gibson

Ubuntu Linux Systems Administrator

JS Gericke Library

Room 1025D

Stellenbosch University

Private Bag X5036

Stellenbosch

7599

South Africa

Tel: +27 21 808 4100 | Cell: +27 84 646 4758

http://scholar.sun.ac.za

http://bit.ly/goodir

http://library.sun.ac.za

http://za.linkedin.com/in/hiltongibson

On 12 June 2014 18:39, Hassan Bhuiyan <Hassan....@brunel.ac.uk> wrote:

Hi

I am trying to upgrade from DSpace1.6.2 to DSpace 4.1. Everything went okay except index-discovery.
I can see community and collection lists and log in to DSpace administrator account and see items in the pool waiting to be archived.

Problem: Homepage shows message "Internal Systems Error" and does not show the item lists.
It will be helpful if someone can help me please.

Note: I tried with rebuild and fresh installation again but still the problem remains same.
Note: I don't see anything directory called "/solr/search" directory in my TOMCAT.

Regards
Hassan Bhuiyan
Brunel University, UK

I get following error while executing index-discovery -f ( when TOMCAT is running)
----------------------------------------------------------------------------------------------------
Exception: Server at http://localhost:8080/solr/search returned non ok status:404, message:Not Found
org.dspace.discovery.SearchServiceException: Server at http://localhost:8080/solr/search returned non ok status:404, message:Not Found
at org.dspace.discovery.SolrServiceImpl.cleanIndex(SolrServiceImpl.java:484)
at org.dspace.discovery.IndexClient.main(IndexClient.java:119)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.dspace.app.launcher.ScriptLauncher.runOneCommand(ScriptLauncher.java:225)
at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:77)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://localhost:8080/solr/search returned non ok status:404, message:Not Found
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:90)
at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:301)
at org.dspace.discovery.SolrServiceImpl.getSolr(SolrServiceImpl.java:155)
at org.dspace.discovery.SolrServiceImpl.cleanIndex(SolrServiceImpl.java:444)
... 7 more

I get following error while executing index-discovery -f ( when TOMCAT is stopped)
------------------------------------------------------------------------------------------------------

Exception: Server refused connection at: http://localhost:8080/solr/search
org.dspace.discovery.SearchServiceException: Server refused connection at: http://localhost:8080/solr/search
at org.dspace.discovery.SolrServiceImpl.cleanIndex(SolrServiceImpl.java:484)
at org.dspace.discovery.IndexClient.main(IndexClient.java:119)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.dspace.app.launcher.ScriptLauncher.runOneCommand(ScriptLauncher.java:225)
at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:77)
Caused by: org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://localhost:8080/solr/search
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:428)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
at org.apache.solr.client.solrj.SolrServer.deleteByQuery(SolrServer.java:285)
at org.apache.solr.client.solrj.SolrServer.deleteByQuery(SolrServer.java:271)
at org.dspace.discovery.SolrServiceImpl.cleanIndex(SolrServiceImpl.java:450)
... 7 more
Caused by: org.apache.http.conn.HttpHostConnectException: Connection to http://localhost:8080 refused
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:190)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:640)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:479)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
... 12 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
... 19 more

------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
DSpace-tech mailing list
DSpac...@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette

------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
DSpace-tech mailing list
DSpac...@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette

--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

Reply all

Reply to author

Forward

0 new messages