Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information)

463 views
Skip to first unread message

George Stanley Kozak

unread,
Aug 26, 2015, 2:22:56 PM8/26/15
to dspac...@lists.sourceforge.net

Hi, Everyone:

 

I have discovered that when I try to optimize my Solr indexes when I upgraded from DSpace 3.3 to 4.3

(wget ‘http://localhost:8080/solr/statistics/update?optimize=true’)

I get the following errors in the solr logs:

ERROR org.apache.solr.core.CoreContainer @ Unable to create core: statistics

Format version is not supported (resource: segment _32 in resource ChecksumIndexInput(MMapIndexInput(path="/cul/app/dspace/solr/statistics/data/index/segments_1p0"))): 2.x. This version of Lucene only supports indexes created with release 3.0 and later.

 

If I later try to upgrade to 5.1, I see during the ant update:

/cul/app/dspace/src/dspace-5.1-src-release/dspace/target/dspace-installer/build.xml:1061: ERROR occurred while checking Solr index version:

Exception in thread "main" java.io.IOException: Could not read Lucene segments files in /cul/app/dspace/solr/statistics/data/index

 

I know the 5.1 documentation states that you can manually update the Solr indexes.  Should that be done in my 4.3 upgrade before I try to go to 5.1 or should it be done in the 5.1 upgrade? 

 

George Kozak

Digital Library Specialist

Cornell University Library Information Technologies (CUL-IT)

218 Olin Library

Cornell University

Ithaca, NY 14853

607-255-8924

 

Tim Donohue

unread,
Aug 26, 2015, 2:23:36 PM8/26/15
to dspac...@lists.sourceforge.net
Hi George,

The error below is essentially saying that your Solr statistics index is
"too old" for the version of Solr packages with DSpace 4.3 to upgrade.

However, in the DSpace 5.1 upgrade process, during the "ant update"
step, we've attempted to catch this scenario automatically -- we try to
determine the version of a Solr index, and upgrade it automatically to
the latest version.

So, I'd recommend trying to point DSpace 5.1 at your old Solr index, and
running "ant update". It is supposed to upgrade an old index automatically.

If that doesn't work for some reason, another option is to do a more
manual upgrade, as described in the DSpace 5.x documentation. This
manual upgrade essentially requires downloading multiple versions of the
Solr/Lucene core JAR, and running it against your index in order to
upgrade it to the latest compatible version (this is essentially the
same process that is now automated by "ant update" though)

https://wiki.duraspace.org/display/DSDOC5x/Upgrading+DSpace#UpgradingDSpace-ManuallyUpgradingSolrIndexes

By the way, from your earlier messages, recently I also did encounter
that odd Solr "write.lock" error ("Index locked for write..") during a
recent upgrade I tried to 5.1. In my situation, it seemed like it was a
stale "write.lock" file which was somehow sitting around. After clearing
it out, the upgrade proceeded. Admittedly, I still need to dig a bit
further and ensure my analysis is correct.

- Tim
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming The Go Parallel Website, sponsored
> by Intel and developed in partnership with Slashdot Media, is your hub for all
> things parallel software development, from weekly thought leadership blogs to
> news, videos, case studies, tutorials and more. Take a look and join the
> conversation now. http://goparallel.sourceforge.net/
>
>
>
> _______________________________________________
> DSpace-tech mailing list
> DSpac...@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
> List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
>

George Stanley Kozak

unread,
Aug 26, 2015, 2:23:38 PM8/26/15
to Tim Donohue, dspac...@lists.sourceforge.net
Tim:

Thank you for your advice. I did try the 5.1 ant update, but it failed. Maybe if I tried it going from DSpace 3.3 or 1.8.2 directly to DSpace 5.1 instead of doing the intermediate step to 4.3, this might work? I will try a few other things and let everyone know what works.

George Kozak
Digital Library Specialist
Cornell University Library Information Technologies (CULIT)
218 Olin Library
Cornell University
Ithaca, NY 14853
607-255-8924

________________________________________
From: Tim Donohue <tdon...@duraspace.org>
Sent: Monday, April 6, 2015 4:44 PM
To: dspac...@lists.sourceforge.net
Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information)
------------------------------------------------------------------------------
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF

Tim Donohue

unread,
Aug 26, 2015, 2:23:45 PM8/26/15
to George Stanley Kozak, dspac...@lists.sourceforge.net
Hi George,

Yes, that is what I was suggesting...upgrading directly from 1.8.2 or
3.3 (whichever was your starting point) to 5.1. The "ant update" should
take care of the necessary Solr index upgrades for you.

- Tim

George Stanley Kozak

unread,
Aug 26, 2015, 2:23:47 PM8/26/15
to Tim Donohue, dspac...@lists.sourceforge.net
Thanks for your help, Tim:

I'm afraid that I keep hitting a wall when I try to upgrade to DSpace 5.1

I tried to go directly from DSpace 1.8.2 to DSpace 5.1 and got:
ERROR occurred while checking Solr index version:
Exception in thread "main" java.io.IOException: Could not read Lucene segments files in /cul/app/dspace/solr/statistics/data/index

We have been using our own stats gathering system instead of the one within DSpace. So, I removed the statistics index but then I got:
Exception in thread "main" java.io.IOException: Could not read Lucene segments files in /cul/app/dspace/solr/search/data/index
This version of Lucene only supports indexes created with release 3.0 and later.

So, I went ahead and used the DSpace 4.3 version of our solr indexes (without the statistics). It ran fine and everything got upgraded, but when I tried to do any searching, I get the "write lock error". If I turn off tomcat, delete the write lock and restart things, I get the same thing. Also, I cannot update the discovery index because of the write lock problem.

By the way, I did try the manual upgrade of the Solr indexes defined in the manual, but that didn't work for me, either.

So, right now, it looks like I can only upgrade to DSpace 4.3, unless you have some more suggestions for me.


George Kozak
Digital Library Specialist
Cornell University Library Information Technologies (CUL-IT)
218 Olin Library
Cornell University
Ithaca, NY 14853
607-255-8924



Tim Donohue

unread,
Aug 26, 2015, 2:23:49 PM8/26/15
to George Stanley Kozak, dspac...@lists.sourceforge.net
Hi George,

What happens if you do the following against your old 1.8.2 indexes:

1. wget
"http://search.maven.org/remotecontent?filepath=org/apache/lucene/lucene-core/3.5.0/lucene-core-3.5.0.jar"

2. java -cp lucene-core-3.5.0.jar org.apache.lucene.index.IndexUpgrader
/cul/app/dspace/solr/statistics/data/index

3. java -cp lucene-core-3.5.0.jar org.apache.lucene.index.IndexUpgrader
/cul/app/dspace/solr/search/data/index

Do either of those "IndexUpgrader" commands succeed against your *old*
1.8.2 indexes? If not, could you send along the exception stacktrace?

I'm just trying to determine if the issue is actually in these old 1.8.2
indexes. It may be either they are indexes from an unexpected, older
version of Solr, or there's some minor index corruption here.

--

To be honest, another thing to note here is that your *search* indexes
can be easily rebuilt in DSpace 5.x (simply run "./dspace
index-discovery -b"). The only older index that really requires this
upgrade process is the *statistics* index (as currently it doesn't have
a reindex command).

So, if your statistics index is of no importance, you could just delete
both of these old indexes and upgrade to 5.1. As part of the 5.1
upgrade, a full reindex of the search index will be kicked off (when you
start Tomcat). So, after a few minutes, your content should be fully
reindexed and working.

- Tim

George Stanley Kozak

unread,
Aug 26, 2015, 2:23:51 PM8/26/15
to Tim Donohue, dspac...@lists.sourceforge.net
Hi, Tim:

1) I did the "lucene-core-3.5.0.jar" manual update of the indexes. For the search index, I got no errors, but for the statistics core, I got:
Exception in thread "main" org.apache.lucene.index.IndexNotFoundException: org.apache.lucene.store.MMapDirectory@/cul/app/dspace/solr/statistics/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2e888d65
at org.apache.lucene.index.IndexUpgrader.upgrade(IndexUpgrader.java:118)
at org.apache.lucene.index.IndexUpgrader.main(IndexUpgrader.java:85)

2) I deleted the statistics core and did the ant update. I brought up my DSpace 5.1 test system and when I tried to do a search, I got no results. When I went to do a browse, I got:
ERROR org.dspace.app.xmlui.aspect.discovery.SidebarFacetsTransformer @ anonymous:session_id=5749DBA88953D8B0ABDDA3FFD1DFF5F2:ip_addr=128.84.117.195:Error in Discovery while setting up date facet range:date facet\colon; dateIssued.year
SolrCore 'search' is not available due to init failure: Index locked for write for core search,trace=org.apache.solr.common.SolrException: SolrCore 'search' is not available due to init failure: Index locked for write for core search

3) I stopped tomcat, deleted the search index (as you suggested) and did the mvn build and ant update and then copied the new webapps over to tomcat. Everything looked good. I started tomcat and my site came up. Before doing anything more I ran "./dspace index-discovery -b". It ran to completion, but all searches return no hits and the browses give me the same write lock error as above.
I made sure the permissions are correct (dspace user owns the solr indexes and tomcat) and removed the write lock and tried again, but again I get the same error.
>> F _______________________________________________

Tim Donohue

unread,
Aug 26, 2015, 2:23:52 PM8/26/15
to George Stanley Kozak, dspac...@lists.sourceforge.net

Hi George,

On 4/7/2015 1:09 PM, George Stanley Kozak wrote:
> Hi, Tim:
>
> 1) I did the "lucene-core-3.5.0.jar" manual update of the indexes. For the search index, I got no errors, but for the statistics core, I got:
> Exception in thread "main" org.apache.lucene.index.IndexNotFoundException: org.apache.lucene.store.MMapDirectory@/cul/app/dspace/solr/statistics/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2e888d65
> at org.apache.lucene.index.IndexUpgrader.upgrade(IndexUpgrader.java:118)
> at org.apache.lucene.index.IndexUpgrader.main(IndexUpgrader.java:85)

This would imply that the "search" index upgraded fine (no errors).

But, the error for your "statistics" index almost implies that it's not
a valid Solr index. Do you use the "statistics" index? Is there
anything in the "/cul/app/dspace/solr/statistics/data/index" directory
or is it empty?


> 2) I deleted the statistics core and did the ant update. I brought up my DSpace 5.1 test system and when I tried to do a search, I got no results. When I went to do a browse, I got:
> ERROR org.dspace.app.xmlui.aspect.discovery.SidebarFacetsTransformer @ anonymous:session_id=5749DBA88953D8B0ABDDA3FFD1DFF5F2:ip_addr=128.84.117.195:Error in Discovery while setting up date facet range:date facet\colon; dateIssued.year
> SolrCore 'search' is not available due to init failure: Index locked for write for core search,trace=org.apache.solr.common.SolrException: SolrCore 'search' is not available due to init failure: Index locked for write for core search
>
> 3) I stopped tomcat, deleted the search index (as you suggested) and did the mvn build and ant update and then copied the new webapps over to tomcat. Everything looked good. I started tomcat and my site came up. Before doing anything more I ran "./dspace index-discovery -b". It ran to completion, but all searches return no hits and the browses give me the same write lock error as above.
> I made sure the permissions are correct (dspace user owns the solr indexes and tomcat) and removed the write lock and tried again, but again I get the same error.

These other two errors seem rather odd to me...like something else is
accessing Solr at the same time in which you are trying to reindex, and
those multiple simultaneous "writes" are throwing errors.

Can you try something even more simplistic than what you've done above?

1. Stop Tomcat
2. Copy your 1.8.3 data over to where you will install DSpace 5
3. Immediately delete *everything* under "/cul/app/dspace/solr/"
(including the entire "statistics" and "search" subdirectories). This
will wipe out all existing indexes, obviously.
4. Build DSpace 5.1 (mvn package). Then run "ant update". This should
complete, as you'll have no existing Solr indexes for it to complain about.
5. Finally, start Tomcat back up.

Now, sit back and wait. Don't run any other commands. But, feel free to
check the logs for any issues as Tomcat boots up.

Behind the scenes, DSpace should *automatically* be reindexing your
content (and recreating the "search" index from nothing). You won't need
to manually run anything (like 'index-discovery -b'), as this is all
automated in DSpace 5 on the first bootup. (After the first bootup
though, 'index-discovery -b' is the way to force a reindex.)

I'm just curious if starting from *completely empty* indexes and
installing DSpace 5 (without running 'index-discovery' manually) will
act any different. If you still hit problems here, I'd recommend sending
us a full stacktrace of the error messages you encounter.

- Tim

George Stanley Kozak

unread,
Aug 26, 2015, 2:23:54 PM8/26/15
to Tim Donohue, dspac...@lists.sourceforge.net
Hi, Tim:

To answer one of your questions: my original DSpace 1.8.2 solr statistics core did have data in it. Attached is a "snapshot" of the index directory.

Now, I did as you asked. I deleted the all of the solr cores (actually everything under /cul/app/dspace/solr)

I did the mvn and ant update and everything went well. I copied the webapps over to tomcat and restarted tomcat. I watched the DSpace log and the catalina.out log. Everything seemed to move along smoothly. When everything was done, I waited a few minutes before I went to my DSpace home page. When my home page came up, I began to see errors in my DSpace log:
ERROR org.dspace.app.xmlui.aspect.discovery.SidebarFacetsTransformer @ anonymous:session_id=B3B12E105B95C8FB8E63E9FB94B5971F:ip_addr=128.84.117.195:Error in Discovery while setting up date facet range:date facet\colon; dateIssued.year
SolrCore 'search' is not available due to init failure: Index locked for write for core search,trace=org.apache.solr.common.SolrException: SolrCore 'search' is not available due to init failure: Index locked for write for core search

Any attempt to do any browse or search results with errors. By the way, the errors on the web appear with " org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Expected mime type application/octet-stream but got text/html." And then raw HTML after that.

What has me stumped is that outside of the problem with the solr statistics, none of these problems appear in my DSpace 4.3 upgrade. I have no write lock problems in DSpace 4.3 install.

George Kozak
Digital Library Specialist
Cornell University Library Information Technologies (CUL-IT)
218 Olin Library
Cornell University
Ithaca, NY 14853
607-255-8924




-----Original Message-----
From: Tim Donohue [mailto:tdon...@duraspace.org]
Sent: Tuesday, April 07, 2015 2:33 PM
To: George Stanley Kozak; dspac...@lists.sourceforge.net
Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information)


Hi George,

solr_stats_index.txt

Brian Freels-Stendel

unread,
Aug 26, 2015, 2:23:55 PM8/26/15
to dspac...@lists.sourceforge.net
Good afternoon,

We saw very similar errors (particularly "Expected mime type application/octet-stream but got text/html") and found that we hadn't updated the appBase directory in tomcat.xml.

Another symptom of ours was the code of the entire error page being written to the dspace log.

It's a longshot, but perhaps something to check.

B--

George Stanley Kozak

unread,
Aug 26, 2015, 2:23:56 PM8/26/15
to Brian Freels-Stendel, dspac...@lists.sourceforge.net
Hi, Brian:

Sorry for being dense, but what do you mean by updating the appBase directory?
------------------------------------------------------------------------------
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT Develop your own process in accordance with the BPMN 2 standard Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_ source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF

Brian Freels-Stendel

unread,
Aug 26, 2015, 2:23:58 PM8/26/15
to George Stanley Kozak, dspac...@lists.sourceforge.net
Hey there,

Sorry for not being descriptive enough. In tomcat.xml, there's a Host section where the DSpace app context fragments go:

___________________________________
<Host name="localhost" appBase="/opt/dspace/webapps"
unpackWARs="true" autoDeploy="true"
xmlValidation="false" xmlNamespaceAware="false">

<!-- DEFINE A CONTEXT PATH FOR DSpace XML User Interface (Manakin) -->
<Context path="/" docBase="/opt/dspace/webapps/xmlui" debug="0" reloadable="true" cachingAllowed="false" allowLinking="true"/>
___________________________________

In that first line, the appBase may still be pointed to Tomcat's default. For us, it let DSpace start and function semi-well, but none of the SOLR stuff would work.

George Stanley Kozak

unread,
Aug 26, 2015, 2:24:08 PM8/26/15
to Brian Freels-Stendel, dspac...@lists.sourceforge.net
Thanks, Brian:

I set my tomcat server.xml as you indicated, but I am still having the same problem with the indexes. I'm not sure what could be wrong. As I stated earlier, everything works fine if I am running DSpace 4.3 and I only have problems once I upgrade to 5.1.

In any case, I thank you and Tim and Hilton and Andrea for your help so far. This problem has stumped me.

Tim Donohue

unread,
Aug 26, 2015, 2:24:09 PM8/26/15
to George Stanley Kozak, dspac...@lists.sourceforge.net
Hi George,

This really has me stumped. It almost starts to sound like a permissions
issue in your index directory (though I know from looking back at this
thread, you said you looked at that).

Since you essentially started with a "fresh" index (no content), and it
still doesn't work, this doesn't sound like an issue with the index
files themselves. While DSpace 5.1 uses a newer version of Solr, it
doesn't act that much different from DSpace 4.3. So, it's confusing to
me that this works for you on DSpace 4.3 and not on DSpace 5.1.

When you encounter that "Index locked for write for core search", there
should be a "write.lock" file sitting in your
/cul/app/dspace/solr/search/data/index/ folder. What are the
permissions on that "write.lock" file? Is it owned by your Tomcat user?
Also is *everything* under /cul/app/dspace/solr/ owned by the Tomcat
user (all files and subdirectories)?

You may even want to try and do the following:

1. Stop Tomcat
2. Recursively change the ownership (just in case), e.g. chown -R
[tomcat-user]:[tomcat-group] /cul/app/dspace/solr/
3. Delete that "write.lock"
4. Start Tomcat again.
5. (You may also need to manually reindex: 'dspace index-discovery -b')

Yea, I know you checked these permissions before, but I'm rather stumped
as to why a fresh, empty index won't even work on your system. I've
never seen that before, and it implies that there may be something lower
level (like a permissions problem) causing problems with Solr.

- Tim

George Stanley Kozak

unread,
Aug 26, 2015, 2:24:15 PM8/26/15
to Tim Donohue, dspac...@lists.sourceforge.net
Hi, Tim:

Yes, this has me stumped.
Here is the permissions on the writelock:

-rw-rw-r-- 1 dspace dspace 0 Apr 8 09:59 write.lock

Tomcat is owned by the dspace user:

drwxr-xr-x 3 dspace dspace 4096 Apr 2 2014 tomcat

Solr is owned by the dspace user:

drwxrwxr-x 6 dspace dspace 4096 Apr 8 09:58 solr

I did as you suggested and stopped tomcat, recursively changed the permissions on the solr indexes:

sudo chown -R dspace:dspace /cul/app/dspace/solr

I deleted the write.lock and restarted tomcat.

I then did /cul/app/dspace/bin/dspace index-discovery -b

Immediately, I received the write lock error and the re-index did not work.

I restored the postgres database and solr indexes to what they were before and did a mvn build and ant update of my DSpace 4.3 source and everything works fine(?!?!)
There is obviously some subtle difference on how DSpace 5.1 and DSpace 4.3 (and lower) are handling the solr indexes and my server set up is just right (or wrong) enough to cause this problem.

George Kozak
Digital Library Specialist
Cornell University Library Information Technologies (CUL-IT)
218 Olin Library
Cornell University
Ithaca, NY 14853
607-255-8924



-----Original Message-----
From: Tim Donohue [mailto:tdon...@duraspace.org]
Sent: Wednesday, April 08, 2015 10:16 AM
To: George Stanley Kozak; dspac...@lists.sourceforge.net
Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information)

Hi George,

This really has me stumped. It almost starts to sound like a permissions issue in your index directory (though I know from looking back at this thread, you said you looked at that).

Since you essentially started with a "fresh" index (no content), and it still doesn't work, this doesn't sound like an issue with the index files themselves. While DSpace 5.1 uses a newer version of Solr, it doesn't act that much different from DSpace 4.3. So, it's confusing to me that this works for you on DSpace 4.3 and not on DSpace 5.1.

When you encounter that "Index locked for write for core search", there should be a "write.lock" file sitting in your /cul/app/dspace/solr/search/data/index/ folder. What are the permissions on that "write.lock" file? Is it owned by your Tomcat user?
Also is *everything* under /cul/app/dspace/solr/ owned by the Tomcat user (all files and subdirectories)?

You may even want to try and do the following:

1. Stop Tomcat
2. Recursively change the ownership (just in case), e.g. chown -R [tomcat-user]:[tomcat-group] /cul/app/dspace/solr/ 3. Delete that "write.lock"
4. Start Tomcat again.
5. (You may also need to manually reindex: 'dspace index-discovery -b')

Yea, I know you checked these permissions before, but I'm rather stumped as to why a fresh, empty index won't even work on your system. I've never seen that before, and it implies that there may be something lower level (like a permissions problem) causing problems with Solr.

- Tim



On 4/7/2015 2:28 PM, George Stanley Kozak wrote:
> Hi, Tim:
>
> To answer one of your questions: my original DSpace 1.8.2 solr statistics core did have data in it. Attached is a "snapshot" of the index directory.
>
> Now, I did as you asked. I deleted the all of the solr cores
> (actually everything under /cul/app/dspace/solr)
>
> I did the mvn and ant update and everything went well. I copied the webapps over to tomcat and restarted tomcat. I watched the DSpace log and the catalina.out log. Everything seemed to move along smoothly. When everything was done, I waited a few minutes before I went to my DSpace home page. When my home page came up, I began to see errors in my DSpace log:
> ERROR org.dspace.app.xmlui.aspect.discovery.SidebarFacetsTransformer @
> anonymous:session_id=B3B12E105B95C8FB8E63E9FB94B5971F:ip_addr=128.84.1

Graham Triggs

unread,
Aug 26, 2015, 2:24:17 PM8/26/15
to George Stanley Kozak, Tim Donohue, dspac...@lists.sourceforge.net
Is it at all possible that you have two separate processes / applications attempting to access the same Solr index - so one locks it, and the other is unable to?

For example, maybe a context file under <tomcat>/config/Catalina/localhost and an application in <tomcat>/webapps?

Regards,
G

George Stanley Kozak

unread,
Aug 26, 2015, 2:24:18 PM8/26/15
to Graham Triggs, Tim Donohue, dspac...@lists.sourceforge.net

Graham:

 

I guess if I had two tomcat processes interfering with each other, I believe it would happen in both my DSpace 4.3 upgrade and my DSpace 5.1 upgrade, but everything works OK under DSpace 4.3 and then breaks down when I go to DSpace 5.1 (no changes to tomcat).   However, I will do a bit more digging.  Perhaps the older DSpace configs are more “forgiving” of problematic tomcat configurations.

 

By the way, I run my webapps under tomcat, but I know that some of you run the webapps in their dspace directory.  Is there any advantage to doing one or another?

Tim Donohue

unread,
Aug 26, 2015, 2:24:24 PM8/26/15
to George Stanley Kozak, Graham Triggs, dspac...@lists.sourceforge.net
Hi George,

On 4/8/2015 1:48 PM, George Stanley Kozak wrote:
> By the way, I run my webapps under tomcat, but I know that some of you
> run the webapps in their dspace directory. Is there any advantage to
> doing one or another?

It's really just a matter of preference, it doesn't matter which way you
load your webapps via Tomcat. But, as Graham notes, the one thing you
don't want to do is have a single webapp loaded *twice*. So, if your
XMLUI (or JSPUI) webapp is both under [tomcat]/webapps/ and configured
under [tomcat]/config/Catalina/localhost/ , then Tomcat may be
essentially running two copies of it. As Graham notes, this also could
be a possible cause of the Solr lock errors.

Out of curiosity, are you running both XMLUI and JSPUI in parallel?
While I've never heard it reported, I wonder if doing so could be
causing that same sort of issue -- one webapp obtains Solr access, and
the other then hits lock errors.

(Again though, this is complete speculation...just trying to determine
what may be unique about your 5.1 setup)

- Tim

George Stanley Kozak

unread,
Aug 26, 2015, 2:24:26 PM8/26/15
to Tim Donohue, Graham Triggs, dspac...@lists.sourceforge.net
Tim and Graham:

Here are excerpts from my tomcat sever.xml file (again, this works OK under DSpace 4.3 and lower):

<Engine name="Catalina" defaultHost="localhost">
...
<Host name="localhost" appBase="webapps"
unpackWARs="true" autoDeploy="true">
...
<Host name="ecommons-test.library.cornell.edu" debug="0"
unpackWARs="true" autoDeploy="false">
...
<Context path="" docBase="[tomcat]/webapps/xmlui" />
...
<Context path="/jspui" docBase="[ tomcat]/webapps/jspui" />

Do you think that this kind of configuration could cause the problem I am observing? And, if so, why doesn't it happen at a lower version of DSpace?

George Kozak
Digital Library Specialist
Cornell University Library Information Technologies (CUL-IT)
218 Olin Library
Cornell University
Ithaca, NY 14853
607-255-8924


-----Original Message-----
From: Tim Donohue [mailto:tdon...@duraspace.org]
Sent: Thursday, April 09, 2015 9:24 AM
To: George Stanley Kozak; Graham Triggs; dspac...@lists.sourceforge.net
Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information)

Tim Donohue

unread,
Aug 26, 2015, 2:24:28 PM8/26/15
to George Stanley Kozak, Graham Triggs, dspac...@lists.sourceforge.net
Hi George,

Hmmm...it actually looks a bit "fishy" to me.

You have two <Host> entries defined, which look to me like they are
loading the same webapps:

1. <Host name="localhost"> is configured to load *everything* under the
"webapps" subdirectory.

2. <Host name="ecommons-test.library.cornell.edu"> is then configured to
load "webapps/xmlui" and "webapps/jspui"

So, essentially, between these two Host definitions, it looks like you
are telling Tomcat to load both the XMLUI and JSPUI *twice*.

Could you try commenting out one of your <Host> definitions and
rebooting Tomcat? I wonder if that would resolve the Solr lock issues.
In general, with the setup you are using, you really only need to either
set "appBase='webapps'" OR define individual <Context> tags per webapp.
You don't need to do both.

As for your other question, I'm not sure why this work work on older
versions of DSpace and not in DSpace 5.1. Perhaps it has something to do
with the fact that we upgraded Solr to the latest version in DSpace 5.1.
Maybe the latest version of Solr is less forgiving about multiple apps
writing to it simultaneously.

- Tim

George Stanley Kozak

unread,
Aug 26, 2015, 2:24:29 PM8/26/15
to Tim Donohue, Graham Triggs, dspac...@lists.sourceforge.net
Tim, et al:

Well, that was it! I have been using the same tomcat config for years, and it appears that it worked OK under older versions of DSpace, but not under 5.1.
I merged all of my stuff under the localhost config and now I no longer get the write lock error.
My DSpace 5.1 is now up and running.

Thank you all!

Mark H. Wood

unread,
Aug 26, 2015, 2:24:39 PM8/26/15
to dspac...@lists.sourceforge.net
On Thu, Apr 09, 2015 at 01:40:55PM +0000, George Stanley Kozak wrote:
> Here are excerpts from my tomcat sever.xml file (again, this works OK under DSpace 4.3 and lower):
>
> <Engine name="Catalina" defaultHost="localhost">
> ...
> <Host name="localhost" appBase="webapps"
> unpackWARs="true" autoDeploy="true">
> ...
> <Host name="ecommons-test.library.cornell.edu" debug="0"
> unpackWARs="true" autoDeploy="false">

Tomcat should be complaining about that Host, since appBase is a
required attribute. I usually just create a separate directory for
each additional Host to point to as its appBase.

I have no idea why the DSpace version should make a difference there.
Probably the difference lies elsewhere. But I would tidy up that Host
declaration, so that I was operating in known territory w.r.t. Tomcat.

--
Mark H. Wood
Lead Technology Analyst

University Library
Indiana University - Purdue University Indianapolis
755 W. Michigan Street
Indianapolis, IN 46202
317-274-0749
www.ulib.iupui.edu
signature.asc

George Stanley Kozak

unread,
Aug 26, 2015, 2:24:40 PM8/26/15
to Mark H. Wood, dspac...@lists.sourceforge.net
Thanks, Mark:

The funny thing is that I have been using the same or similar server.xml file for years with no problems. My theory is that the newest Solr in DSpace 5.1 was more sensitive to my "funny" config, and the older ones were more forgiving.
In any case, I am up and running now and my thanks to you and everyone else for their help and suggestions.

George Kozak
Digital Library Specialist
Cornell University Library Information Technologies (CUL-IT)
218 Olin Library
Cornell University
Ithaca, NY 14853
607-255-8924


-----Original Message-----
From: Mark H. Wood [mailto:mw...@IUPUI.Edu]
Sent: Friday, April 10, 2015 9:15 AM
To: dspac...@lists.sourceforge.net
Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information)

Reply all
Reply to author
Forward
0 new messages