Re: [Dspace-tech] Java fatal error on dspace import

[André]

Hi Andrea, Jose and Mark. Thank you!

I tried Jose suggestion, importing one by one, but the error seemed to be randomic.
I switched to sun java 1.6, but at the same time I reset the dabatase (it was on our test installation), and the problem was gone.

The problem could have been Java7 but could also have been the database, I should have tested them separatedly.
But at least these messages could be a good tip for one who finds the same problem as me in the future.

Thanks again
André Assada

Em 14 de outubro de 2011 17:15, Andrea Bollini <bol...@cilea.it> escreveu:

Hi André,
I noted that you use java 7 I have not direct experience with this but there are a lot of post in the web reporting issues using java 7 with lucene/solr.
See for example: http://www.infoq.com/news/2011/08/java7-hotspot
Hope this help,
Andrea


Il 14/10/2011 19:44, André ha scritto:

Dear all,

I'm trying to import 157 registries on dspace 1.6.2 by calling
[dspace]/bin/dspace import --add --eperson=andre....@usp.br --collection=123456789/32 --source=/home/andre/xImpAleph/impTeste111014/xvi_fd --mapfile=./xvi_fd --workflow

It starts the process ok, but int the middle I get the following error message:


#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007fea376dc440, pid=20001, tid=140644013197072
#
# JRE version: 7.0-b147
# Java VM: Java HotSpot(TM) 64-Bit Server VM (21.0-b17 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# J  org.apache.lucene.index.DocumentsWriter$ThreadState$FieldData.invertField(Lorg/apache/lucene/document/Fieldable;Lorg/apache/lucene/analysis/Analyzer;I)V
#
# Core dump written. Default location: /dspace/bin/core or core.20001 (max size 1 kB). To ensure a full core dump, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /dspace/bin/hs_err_pid20001.log
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.sun.com/bugreport/crash.jsp
#
./dspace: line 69: 20001 Aborted                 java $JAVA_OPTS -classpath $FULLPATH $LOG org.dspace.app.launcher.ScriptLauncher "$@"





If I retry to import, with the --resume option, it restarts very slowly, and in dspace.log I get the following message:





2011-10-14 14:01:26,342 ERROR org.dspace.search.DSIndexer @ Lock obtain timed out: SimpleFSLock@/dspace/search/write.lock
org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: SimpleFSLock@/dspace/search/write.lock
        at org.apache.lucene.store.Lock.obtain(Lock.java:85)
        at org.apache.lucene.index.IndexWriter.init(IndexWriter.java:691)
        at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:452)
        at org.dspace.search.DSIndexer.openIndex(DSIndexer.java:781)
        at org.dspace.search.DSIndexer.writeDocument(DSIndexer.java:853)
        at org.dspace.search.DSIndexer.buildDocument(DSIndexer.java:1138)
        at org.dspace.search.DSIndexer.indexContent(DSIndexer.java:299)
        at org.dspace.search.DSIndexer.updateIndex(DSIndexer.java:584)
        at org.dspace.search.DSIndexer.main(DSIndexer.java:545)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:212)




Searching the archive of this list, I found some people solved this by deleting the write.lock and afterwards force the reindexation by running
./dsrun org.dspace.search.DSIndexer -c


This solves the slowdown problem but doesn't solve the import problem.
I tried to stop tomcat before importing, to guarantee none was accessing the index at the same time, but this didn't solve the problem.
I also set more free memory with JAVA_OPTS=-Xmx512m   and also -Xmx1024m, but this also didn't do the trick.

Has anyone had this problem? Could share any ideas?

Thanks in advance

Andre Assada

------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure contains a
definitive record of customers, application performance, security
threats, fraudulent activity and more. Splunk takes this data and makes
sense of it. Business sense. IT sense. Common sense.
http://p.sf.net/sfu/splunk-d2d-oct

_______________________________________________
DSpace-tech mailing list
DSpac...@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

-- 
Dott. Andrea Bollini

bol...@cilea.it
ph. +39 06 59292853 - mob. +39 348 8277525 - fax +39 06 5913770
CILEA - Consorzio Interuniversitario
http://www.cilea.it/disclaimer

------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure contains a
definitive record of customers, application performance, security
threats, fraudulent activity and more. Splunk takes this data and makes
sense of it. Business sense. IT sense. Common sense.
http://p.sf.net/sfu/splunk-d2d-oct
_______________________________________________
DSpace-tech mailing list
DSpac...@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

[Blanco, Jose]

Andre, Just read your post again, and see you already have the lock file problem solved. 

 

I’m not sure what could be happening.  I would try to import one item at a time and see if one in particular is causing the problem and then look into why.

 

-Jose

 

From: Blanco, Jose [mailto:bla...@umich.edu]
Sent: Friday, October 14, 2011 2:28 PM
To: André; dspace-tech
Subject: Re: [Dspace-tech] Java fatal error on dspace import

 

Andre,

 

The Lock error you are getting when you resume is because there is a write.lock file in your search repository directory.  So go into

 

[dspace]/search

 

And you should see a write.lock file in there.  Take a look at it.  I think it should be empty. It is just used to stop indexing from taking place.  If you remove it, you should be able to keep going.  Are you doing this in the live system, or dev area.  I always load things in dev before going to the live, just in case.

 

-Jose

[Mark H. Wood]

I can't explain the main problem, but I think that the slow startup
upon resuming is caused by waiting for the lock file to go away. I
presume that it never goes away because it was left behind when the
initial attempt failed so catastrophically.

--
Mark H. Wood, Lead System Programmer mw...@IUPUI.Edu
Asking whether markets are efficient is like asking whether people are smart.

[André]

[Blanco, Jose]

Andre,

 

The Lock error you are getting when you resume is because there is a write.lock file in your search repository directory.  So go into

 

[dspace]/search

 

And you should see a write.lock file in there.  Take a look at it.  I think it should be empty. It is just used to stop indexing from taking place.  If you remove it, you should be able to keep going.  Are you doing this in the live system, or dev area.  I always load things in dev before going to the live, just in case.

 

-Jose

 

From: André [mailto:andre....@usp.br]
Sent: Friday, October 14, 2011 1:45 PM
To: dspace-tech
Subject: [Dspace-tech] Java fatal error on dspace import

 

Dear all,

[Andrea Bollini]

Hi André,
I noted that you use java 7 I have not direct experience with this but there are a lot of post in the web reporting issues using java 7 with lucene/solr.
See for example: http://www.infoq.com/news/2011/08/java7-hotspot
Hope this help,
Andrea


Il 14/10/2011 19:44, André ha scritto:

------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure contains a
definitive record of customers, application performance, security
threats, fraudulent activity and more. Splunk takes this data and makes
sense of it. Business sense. IT sense. Common sense.
http://p.sf.net/sfu/splunk-d2d-oct

_______________________________________________
DSpace-tech mailing list
DSpac...@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech