DS 7.2 - Solr Statistics not updating

[John Fitzgerald]

Hey all, 

I recently started evaluating DS 7.2 and am excitedly anxious to migrate my DS 6 instance. 

This DS 7.2 instance is pretty much an out-of-the-box install, and I'm very pleased with how it's running, and I successfully added a few diverse items in a single collection. However, it appears that its Solr Statistics are not updating and I'm flummoxed.  Solr Search is working fine, and I don't see any indications that there's an issue with my Solr-instance:

Image of my Solr-dashboard.

According to the official SOLR Statistics instructions:

Unlike previous versions, enabling statistics in DSpace does not require additional installation or customization. 

So I'm thinking I may have in fact messed up a configuration-setting somewhere.

Can anyone advise?

Thanks,

-John

[John Fitzgerald]

Incidentally, I just tired manually reindexing this core, and received this error:

[john@server bin]# ./dspace solr-reindex-statistics
No index name provided, defaulting to "statistics".
Exception: Error from server at http://localhost:8983/solr: Path /dspace7/solr/statistics must be relative to SOLR_HOME, SOLR_DATA_HOME coreRootDirectory. Set system property 'solr.allowPaths' to add other allowed paths.
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://localhost:8983/solr: Path /dspace7/solr/statistics must be relative to SOLR_HOME, SOLR_DATA_HOME coreRootDirectory. Set system property 'solr.allowPaths' to add other allowed paths.
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:681)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:266)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:214)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:231)
        at org.dspace.util.SolrImportExport.reindex(SolrImportExport.java:320)
        at org.dspace.util.SolrImportExport.main(SolrImportExport.java:189)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.dspace.app.launcher.ScriptLauncher.runOneCommand(ScriptLauncher.java:277)
        at org.dspace.app.launcher.ScriptLauncher.handleScript(ScriptLauncher.java:133)
        at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:98)

[Tim Donohue]

Hi John,

Have you checked your logs whenever performing an action in DSpace 7 which should be generating statistics?  E.g. viewing a Community/Collection/Item, performing a search etc?  It's possible some error is being reported there which can lead you to what the core problem may be.   I'd recommend checking the dspace.log first, but if nothing useful is there, also check Tomcat and Solr logs just in case they are reporting an issue.

You also could double check that you copied all the DSpace Solr Core configs over to Solr during the install... see step 13 here: https://wiki.lyrasis.org/display/DSDOC7x/Installing+DSpace#InstallingDSpace-InstallingtheBackend(ServerAPI)   If somehow you are missing some of the files for the "statistics" core, that would cause odd behavior.

If you find anything else, let us know on this list.

Tim

[John Fitzgerald]

Thank you for your prompt response, Tim! It is truly appreciated. I will check this out tonight.

After my posting, I stumbled across this:

https://groups.google.com/g/dspace-tech/c/o2qou4tgb4E/m/ySM4oaV1BgAJ

The author mentions that ./dspace solr-reindex-statistics in DS7 is "broken" and to avoid using it. Is this accurate?

Thanks,

-John

[DSpace Community]

Hi John,

Thanks for making that connection! I didn't realize that the "./dspace solr-reindex-statistics" script wasn't working properly, as no one had reported that in our ticketing system yet. But I can also verify it is broken on my end. So, I created a ticket to fix it: https://github.com/DSpace/DSpace/issues/8181

I appreciate you pointing that out, as when no one creates a ticket, then the bug may go unnoticed by the core developers (as much as we try to follow the messages in the mailing lists, sometimes things get "lost" in email).   So, just as a tip in the future... you and everyone else on this list are more than welcome to report things that look like bugs in https://github.com/DSpace/DSpace/issues .  That ensures it gets on the developers "plate" and we'll try to see if it really is a bug (and if so, find a fix).

Thanks,

Tim

[John Fitzgerald]

Hey Tim, 

Well, I must say, it's nice to be inadvertently helpful!

Thanks for your continued and excellent support.

I'm actually in the process of piecemeal testing an existing DS 6.4 instance's (meta)data against this test DS 7.3 instance. I wanted to set up a parallel DS 6.4 using the same Tomcat & Solr servers as its twin; Tomcat's easy for paralleling, but I'd like to use different core-names in Solr; I started configuring a second Solr instance on the same server, but the scope-creep discouraged me. Is there a setting in DS 6's configs where one can assign different core-names to delineate between the DS instances? Incidentally, about 20% of this effort is basically my exercising my DSpace muscles; I'd like to know DS 6 as inside-out as practicable before moving a very large and busy repository over to DS 7. I inherited the DS 6 instance, and it's needed very little reconfiguration over the years. I think I know more about the guts of DS 7 than DS 6 at this point!

 

Thanks,

-John

[John Fitzgerald]

OOPS!

*DS 7.2

[Mark H. Wood]

On Sun, Feb 20, 2022 at 12:51:28PM -0800, John Fitzgerald wrote:
> I'm actually in the process of piecemeal testing an existing DS 6.4
> instance's (meta)data against this test DS 7.3 instance. I wanted to set up
> a parallel DS 6.4 using the same Tomcat & Solr servers as its twin;
> Tomcat's easy for paralleling, but I'd like to use different core-names in
> Solr; I started configuring a second Solr instance on the same server, but
> the scope-creep discouraged me. Is there a setting in DS 6's configs where
> one can assign different core-names to delineate between the DS instances?

There is nothing to facilitate this in DSpace 6, but it shouldn't be
difficult. I've been doing something similar in my development
environment for some time.

There is a pending patch for DSpace 7 at
https://github.com/DSpace/DSpace/pull/8147 that shows a way to do this
in coordinated fashion. DSpace 6 would be quite similar.

Another way would be to dispense with the prefix property and just
edit the core names individually in the indicated configuration files.

DSpace doesn't care how the cores are named. You just need agreement
between Solr and DSpace.

--
Mark H. Wood
Lead Technology Analyst

University Library
Indiana University - Purdue University Indianapolis
755 W. Michigan Street
Indianapolis, IN 46202
317-274-0749
www.ulib.iupui.edu