DSpace 6.2 - statistics maintenance problems

[Evgeni Dimitrov]

I have not done any statistics maintenance for several years. Data is growing and I did:

1. stopped DSpace and backed up /dspace/DSPACE/solr/statistics/data/

2. optimize:
 ./dspace stats-util -o
SOLR Optimize -- Process Started:1570563559015
Exception: background merge hit exception: _3epm(4.10.4):C5192613 _2nyj(4.10.4):C4234562

The whole exception is in the end

3. split by years:
./dspace stats-util -s
Moving: 437373 into core statistics-2016
Moving: 4822733 into core statistics-2017
Exception: No space left on device
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: No space left on device

This created directories statistics-2016, statistics-2017 and hundreds of tempxxx.csv files in the DSPACE/temp directory

4. I deleted the directories and the files. Then DSpace created again statistics-2017 with some data files in it.

5. I stopped DSpace, deleted statistics-2017, /dspace/DSPACE/solr/statistics/data/ and restored the initial /dspace/DSPACE/solr/statistics/data/

6. Now statistics does not work. The log file says
2019-10-09 08:37:21,160 ERROR org.apache.solr.core.CoreContainer @ Error creating core [statistics]: Error opening new searcher
org.apache.solr.common.SolrException: Error opening new searcher
. . .
Caused by: java.nio.file.NoSuchFileException: /dspace/DSPACE/solr/statistics/data/index/segments_21us

My concern is:
1. How to make the statistics work again?

2. How does "stats-util -s" work? I mean - If I provide more disk space for the DSPACE/temp directory, will this be enough for the "stats-util -s" to finish successfully? Does it move everything temporarily to DSPACE/temp? Or should there be space for some overlap of the old statistics and the new populated statistics-2016, statistics-2017, ...?

Best regards

=========== the exception during optimize:
[root@dspace6 bin]# ./dspace stats-util -o
SOLR Optimize -- Process Started:1570563559015
Exception: background merge hit exception: _3epm(4.10.4):C5192613 _2nyj(4.10.4):C4234562 _1yg5(4.10.4):C3952629 _xme(4.10.2):C2658195 _3yb6(4.10.4):C2239573 _3mtu(4.10.4):C598509 _3hb8(4.10.4):C448155 into _3ybb [maxNumSegments=1]
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: background merge hit exception: _3epm(4.10.4):C5192613 _2nyj(4.10.4):C4234562 _1yg5(4.10.4):C3952629 _xme(4.10.2):C2658195 _3yb6(4.10.4):C2239573 _3mtu(4.10.4):C598509 _3hb8(4.10.4):C448155 into _3ybb [maxNumSegments=1]
        at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:552)
        at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
        at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
        at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:124)
        at org.apache.solr.client.solrj.SolrServer.optimize(SolrServer.java:204)
        at org.apache.solr.client.solrj.SolrServer.optimize(SolrServer.java:158)
        at org.dspace.statistics.SolrLoggerServiceImpl.optimizeSOLR(SolrLoggerServiceImpl.java:1161)
        at org.dspace.statistics.util.StatisticsClient.main(StatisticsClient.java:96)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.dspace.app.launcher.ScriptLauncher.runOneCommand(ScriptLauncher.java:229)
        at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:81)
[root@dspace6 bin]#