[Dspace-tech] Solr Service Unavailable - Statistics

[Antoanne Christopher]

After update my DSpace instance [ http://virtualbib.fgv.br/dspace ] from 1.5.2 to 1.6.0 and start to user solr statistics, I have received several erros on dspace.log with message "Service Unavailable" about solr exception, .

Someone has seen this error?
Know how to configure the solr transacions log to see exactly waht is happening during the solr transactions?

====

2010-05-07 17:28:15,500 ERROR org.dspace.statistics.SolrLogger @ Service Unavailable

Service Unavailable

request: http://virtualbib.fgv.br/solr/statistics/update
org.apache.solr.common.SolrException: Service Unavailable

Service Unavailable

request: http://virtualbib.fgv.br/solr/statistics/update
        at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:343)
        at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:183)
        at org.apache.solr.client.solrj.request.UpdateRequest.process(UpdateRequest.java:217)
        at org.apache.solr.client.solrj.SolrServer.commit(SolrServer.java:85)
        at org.dspace.statistics.SolrLogger.post(SolrLogger.java:246)
        at org.dspace.statistics.SolrLoggerUsageEventListener.receiveEvent(SolrLoggerUsageEventListener.java:41)
        at org.dspace.services.events.SystemEventService.fireLocalEvent(SystemEventService.java:154)
        at org.dspace.services.events.SystemEventService.fireEvent(SystemEventService.java:97)
        at org.dspace.app.xmlui.cocoon.BitstreamReader.setup(BitstreamReader.java:372)
        at sun.reflect.GeneratedMethodAccessor45.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.cocoon.core.container.spring.avalon.PoolableProxyHandler.invoke(PoolableProxyHandler.java:71)
        at $Proxy29.setup(Unknown Source)
        at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.setupReader(AbstractProcessingPipeline.java:560)
        at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.preparePipeline(AbstractProcessingPipeline.java:464)
        at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.process(AbstractProcessingPipeline.java:411)
        at sun.reflect.GeneratedMethodAccessor99.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.cocoon.core.container.spring.avalon.PoolableProxyHandler.invoke(PoolableProxyHandler.java:71)
        at $Proxy28.process(Unknown Source)
        at org.apache.cocoon.components.treeprocessor.sitemap.ReadNode.invoke(ReadNode.java:94)
        at org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:78)
        at org.apache.cocoon.components.treeprocessor.sitemap.SelectNode.invoke(SelectNode.java:87)
        at org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:55)
        at org.apache.cocoon.components.treeprocessor.sitemap.MatchNode.invoke(MatchNode.java:87)
        at org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:55)
        at org.apache.cocoon.components.treeprocessor.sitemap.MatchNode.invoke(MatchNode.java:87)
        at org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:78)
        at org.apache.cocoon.components.treeprocessor.sitemap.PipelineNode.invoke(PipelineNode.java:143)
        at org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:78)
        at org.apache.cocoon.components.treeprocessor.sitemap.PipelinesNode.invoke(PipelinesNode.java:81)
        at org.apache.cocoon.components.treeprocessor.ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:239)
        at org.apache.cocoon.components.treeprocessor.ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:171)
        at org.apache.cocoon.components.treeprocessor.TreeProcessor.process(TreeProcessor.java:247)
        at org.apache.cocoon.servlet.RequestProcessor.process(RequestProcessor.java:351)
        at org.apache.cocoon.servlet.RequestProcessor.service(RequestProcessor.java:169)
        at org.apache.cocoon.servlet.SitemapServlet.service(SitemapServlet.java:42)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
        at sun.reflect.GeneratedMethodAccessor108.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:310)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:77)
        at org.apache.cocoon.jnet.URLHandlerFactoryCollector.installURLHandlers(URLHandlerFactoryCollector.java:37)
        at sun.reflect.GeneratedMethodAccessor107.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:627)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:616)
        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:64)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:89)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
        at $Proxy13.service(Unknown Source)
        at org.apache.cocoon.servletservice.ServletServiceContext$PathDispatcher.forward(ServletServiceContext.java:481)
        at org.apache.cocoon.servletservice.ServletServiceContext$PathDispatcher.forward(ServletServiceContext.java:455)
        at org.apache.cocoon.servletservice.spring.ServletFactoryBean$ServiceInterceptor.invoke(ServletFactoryBean.java:245)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
        at $Proxy14.service(Unknown Source)
        at org.apache.cocoon.servletservice.DispatcherServlet.service(DispatcherServlet.java:106)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.dspace.app.xmlui.cocoon.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:142)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.dspace.app.xmlui.cocoon.DSpaceCocoonServletFilter.doFilter(DSpaceCocoonServletFilter.java:257)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.cocoon.servlet.multipart.MultipartFilter.doFilter(MultipartFilter.java:131)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.dspace.utils.servlet.DSpaceWebappServletFilter.doFilter(DSpaceWebappServletFilter.java:112)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
        at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:190)
        at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:283)
        at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:767)
        at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:697)
        at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:889)
        at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)
        at java.lang.Thread.run(Thread.java:619)

====

Thanks for any guidance on this
Antoanne Pontes

[Mark Diggory]

You will need to change a setting in your configuration so that dspace
1.6.0 is looking in the correct place for solr

Try the following in your dspace.cfg
solr.log.server = http://localhost:8080/solr/statistics

or if oyu are not running port 8080 on your tomcat engine, try

solr.log.server = http://localhost/solr/statistics

We are aware of the issue and planning to adjust this in dspace 1.6.1

Mark

--
Mark R. Diggory
Head of U.S. Operations - @mire

http://www.atmire.com - Institutional Repository Solutions
http://www.togather.eu - Before getting together, get Tog@ther

[Antoanne Christopher]

Helo Mark!

The error continues after this setting.

I had already configured the hosts file server for my domain to resolve as 127.0.0.1.

Do you have any other suggestions?

See my log file after make your suggestion:

request: http://localhost:8080/solr/statistics/update

org.apache.solr.common.SolrException: Service Unavailable

Service Unavailable

request: http://localhost:8080/solr/statistics/update

        at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:343)

        at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:183)

        at org.apache.solr.client.solrj.request.UpdateRequest.process(UpdateRequest.java:217)

        at org.apache.solr.client.solrj.SolrServer.commit(SolrServer.java:85)

        at org.dspace.statistics.SolrLogger.post(SolrLogger.java:246)

        at org.dspace.statistics.SolrLoggerUsageEventListener.receiveEvent(SolrLoggerUsageEventListener.java:41)

        at org.dspace.services.events.SystemEventService.fireLocalEvent(SystemEventService.java:154)

        at org.dspace.services.events.SystemEventService.fireEvent(SystemEventService.java:97)

        at org.dspace.app.xmlui.cocoon.BitstreamReader.setup(BitstreamReader.java:372)

        at sun.reflect.GeneratedMethodAccessor53.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

        at java.lang.reflect.Method.invoke(Method.java:597)

        at org.apache.cocoon.core.container.spring.avalon.PoolableProxyHandler.invoke(Pool........

Thanks!!

Antoanne Pontes

[Mark Diggory]

First, I would verify if the solr server is actually running locally
on the machine via wget, curl or a browser local to that machine.

Mark

[Gilbertson, Keith R]

A colleague in charge of administering a larger repository mentioned seeing these errors on Friday.

I looked through the log files on a smaller set of test systems that I've been using, and I also have them.  They occur infrequently, maybe 2-3 times in a 24 hour period.  The stats updates work most of the time, but it looks like every once in a while the update fails.

I haven't been able to cause them to happen on my systems at will - they just happen occasionally.  On his system, which receives substantial traffic, he can cause the errors by visiting an item page and rapidly clicking and re-clicking the refresh button on the browser.  I think that he was going to adjust some solr settings to see if that fixed the error.

If I hear more about this or have a chance to investigate some more I'll let you know.  Please do the same if you find a solution!

--keith


 
----- Mensaje original -----
De: "Antoanne Christopher" <anto...@gmail.com>
Para: dspac...@lists.sourceforge.net
Enviados: Viernes, 7 de Mayo 2010 16:53:59 GMT -05:00 Región oriental EE. UU./Canadá
Asunto: [Dspace-tech] Solr Service Unavailable - Statistics

------------------------------------------------------------------------------
_______________________________________________ DSpace-tech mailing list DSpac...@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech

[Mark Diggory]

I would recommend creating a JIRA ticket to track the subject/issue.
I think we will want to look into mediating the number of updated that
occur during a user session for any one Item View. This will mean
keeping a cache of visited pages in the session and assuring that item
visits are recorded but once a session.

Mark

[Antoanne Christopher]

Thanks Mark!
Now I created a ticket in jira, see DS-576.

Antoanne

--
Att,
Antoanne Pontes
http://antoanne.com - Internet Design