RE: Unable to import marc after 1.3 upgrade
Demian Katz
I’m copying this to solrmarc-tech in case anyone there has any ideas….
Do you still have a copy your 1.2 installation? If you run import-march.sh in 1.2, does the java command line displayed there differ significantly from the output of the 1.3 version?
I’m not sure what could be going wrong here – the java command should be pointing to SolrMarc.jar, which should contain all the necessary solrmarc code; I don’t think it’s a CLASSPATH problem since it’s not an external library that’s missing… though I’m far from being a Java expert!
Anyway, maybe if you can provide a few more details we can offer some ideas….
- Demian
From: Bernhardt, Russell (CIV) [mailto:rgbe...@nps.edu]
Sent: Wednesday, February 08, 2012 7:45 PM
To: vufind-...@lists.sourceforge.net
Subject: [VuFind-General] Unable to import marc after 1.3 upgrade
I think my CLASSPATH isn’t getting set or something, but after attempting to upgrade from 1.2 to 1.3, I can’t reimport our MARC data. Solr fails with a ClassNotFoundException, saying that it can’t find org.solrmarc.marc.MarcImporter.
I tested the upgrade on my development machine and it went just fine with the exact same marc data.
I’m running on RHEL 5.7 (fully patched). Any ideas? I can see the SolrMarc.jar in import, but I normally haven’t had to modify my CLASSPATH to point to it…
Any ideas are greatly appreciated, thanks!
Russ Bernhardt
Systems Analyst
Library Information Systems
Naval Postgraduate School, Monterey CA
Robert J. Haschart
avoid any possibility of CLASSPATH issues and jar conflicts, by using the
one-jar tool to bundle all of the needed jars into one large jar of jars,
with its own internal Class loader.
Additionally nothing has changed in SolrMarc that would cause an
incompatibilty with the import script.
I'm at a conference now (yay Code4lib). I may have some time to look into
it later today, but won't actually be back in the office until Monday.
-Bob Haschart
On Thu, 9 Feb 2012 09:35:02 -0500
Demian Katz <demia...@villanova.edu> wrote:
> I'm copying this to solrmarc-tech in case anyone there has any ideas....
>
> Do you still have a copy your 1.2 installation? If you run
>import-march.sh in 1.2, does the java command line displayed there differ
>significantly from the output of the 1.3 version?
>
> I'm not sure what could be going wrong here - the java command should be
>pointing to SolrMarc.jar, which should contain all the necessary solrmarc
>code; I don't think it's a CLASSPATH problem since it's not an external
>library that's missing... though I'm far from being a Java expert!
>
> Anyway, maybe if you can provide a few more details we can offer some
>ideas....
>
> - Demian
>
>From: Bernhardt, Russell (CIV) [mailto:rgbe...@nps.edu]
> Sent: Wednesday, February 08, 2012 7:45 PM
> To: vufind-...@lists.sourceforge.net
> Subject: [VuFind-General] Unable to import marc after 1.3 upgrade
>
> I think my CLASSPATH isn't getting set or something, but after attempting
>to upgrade from 1.2 to 1.3, I can't reimport our MARC data. Solr fails with
>a ClassNotFoundException, saying that it can't find
>org.solrmarc.marc.MarcImporter.
>
> I tested the upgrade on my development machine and it went just fine with
>the exact same marc data.
>
> I'm running on RHEL 5.7 (fully patched). Any ideas? I can see the
>SolrMarc.jar in import, but I normally haven't had to modify my CLASSPATH
>to point to it...
>
> Any ideas are greatly appreciated, thanks!
>
> Russ Bernhardt
> Systems Analyst
> Library Information Systems
> Naval Postgraduate School, Monterey CA
>
> --
> You received this message because you are subscribed to the Google Groups
>"solrmarc-tech" group.
> To post to this group, send email to solrma...@googlegroups.com.
> To unsubscribe from this group, send email to
>solrmarc-tec...@googlegroups.com.
>For more options, visit this group at
>http://groups.google.com/group/solrmarc-tech?hl=en.
>
Demian Katz
That ZipException is definitely significant – for whatever reason, Java is unable to open up the .jar it needs. A quick Google search for the error turned up a number of discussions. This one, for example, has some ideas that might help:
http://stackoverflow.com/questions/325202/java-util-zip-zipexception-error-in-opening-zip-file
- If Java is having trouble writing to the temp directory, you might see this. (Does using “sudo” change the behavior?)
- Sometimes if you unzip and rezip the offending .jar, you can fix the problem (though the trick here is figuring out which jar is failing – but based on the ClassNotFoundException, perhaps MarcImporter.jar is the culprit).
Good luck, and let us know if you need more ideas!
- Demian
From: Bernhardt, Russell (CIV) [mailto:rgbe...@nps.edu]
Sent: Friday, February 10, 2012 6:07 PM
To: Demian Katz; vufind-...@lists.sourceforge.net; solrma...@googlegroups.com
Subject: RE: Unable to import marc after 1.3 upgrade
Hi Demian and solrmarc mailing list,
Here is the first few lines of the VuFind 1.2 import:
/usr/local/vufind/solr /usr/local/vufind
Now Importing /tmp/full_catalog.mrc ...
/usr/lib/jvm/java-openjdk/bin/java -Xms512m -Xmx512m -Dsolrmarc.solr.war.path=/usr/local/vufind/solr/jetty/webapps/solr.war -Dsolr.core.name=biblio -Dsolrmarc.path=/usr/local/vufind/import -Dsolr.path=/usr/local/vufind/solr -Dsolr.solr.home=/usr/local/vufind/solr -jar /usr/local/vufind/import/SolrMarc.jar import.properties /tmp/full_catalog.mrc
Feb 8, 2012 11:54:36 PM org.solrmarc.marc.MarcImporter main
INFO: Starting SolrMarc indexing.
…
The log continues through successful indexing.
Here is the first few lines of the 1.3 import (using the same marc data):
/usr/local/vufind/solr /usr/local/vufind
Now Importing /tmp/full_catalog.mrc ...
/usr/lib/jvm/java-openjdk/bin/java -Xms512m -Xmx512m -Dsolrmarc.solr.war.path=/usr/local/vufind/solr/jetty/webapps/solr.war -Dsolr.core.name=biblio -Dsolrmarc.path=/usr/local/vufind/import -Dsolr.path=/usr/local/vufind/solr -Dsolr.solr.home=/usr/local/vufind/solr -jar /usr/local/vufind/import/SolrMarc.jar import.properties /tmp/full_catalog.mrc
Unable to load resource: java.util.zip.ZipException: error in opening zip file
java.util.zip.ZipException: error in opening zip file
at java.util.zip.ZipFile.open(Native Method)
at java.util.zip.ZipFile.<init>(ZipFile.java:131)
at java.util.jar.JarFile.<init>(JarFile.java:150)
at java.util.jar.JarFile.<init>(JarFile.java:114)
at com.simontuffs.onejar.JarClassLoader.loadJar(JarClassLoader.java:681)
at com.simontuffs.onejar.JarClassLoader.load(JarClassLoader.java:506)
at com.simontuffs.onejar.JarClassLoader.load(JarClassLoader.java:382)
at com.simontuffs.onejar.Boot.run(Boot.java:310)
at com.simontuffs.onejar.Boot.main(Boot.java:170)
Exception in thread "main" java.lang.ClassNotFoundException: org.solrmarc.marc.MarcImporter
at com.simontuffs.onejar.JarClassLoader.findClass(JarClassLoader.java:946)
at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
at com.simontuffs.onejar.Boot.run(Boot.java:328)
at com.simontuffs.onejar.Boot.main(Boot.java:170)
This is the full output of import-marc.sh. I don’t know if that sheds any light, but I was mostly drawn to the Zip exception and then the ClassNotFoundException.
We’re using RHEL 5, with OpenJDK 1.6.0_20. I tried installing Oracle’s JRE 1.6.0_30 and got the same error.
Any ideas are greatly appreciated. Let me know if you need more info.
Thanks,
Demian Katz
Is /opt/vufind/solr/jetty/webapps/solr.war actually the correct path to your solr.war? (I’m guessing it probably is, but since /opt isn’t the most common place to put VuFind, it seemed worth asking).
Have you tried setting the JETTY_CONSOLE environment variable to a file path so you can capture Solr’s output during the vufind.sh startup? Perhaps there is a clue in there.
Of course, Solr doesn’t actually need to be running for SolrMarc to index data (it can access the .war directly), so the fact that Solr isn’t running isn’t the direct cause of the problem – but whatever is preventing it from starting up is quite possibly related.
- Demian
From: Bernhardt, Russell (CIV) [mailto:rgbe...@nps.edu]
Sent: Monday, February 13, 2012 7:01 PM
To: Demian Katz; vufind-...@lists.sourceforge.net; solrma...@googlegroups.com
Subject: RE: Unable to import marc after 1.3 upgrade
Hi Demian,
Thanks for the tips. Unfortunately, they didn’t work. I even copied the SolrMarc.jar from our known working 1.2 install and it throws the same errors. I also tested with Oracle’s latest 1.6.30 binary (even removed the local OpenJDK install) and it still fails.
I looked a little deeper and I think it might be an issue with Solr. I tried to start jetty manually and it was getting the ZipException, but just above the ZipException it said:
2012-02-13 07:30:09.108::WARN: Failed startup of context org.mortbay.jetty.webapp.WebAppContext@8f0c85e{/solr,jar:file:/opt/vufind/solr/jetty/webapps/solr.war!/}
After manually starting jetty (it still ran even after the ZipException), I see the jetty temp structure in /tmp/ and I can access the jetty server with localhost:8080, but instead of the 404 error that I get on the healthy 1.2 system, I get 503 service unavailable which leads me to believe Solr isn’t starting up. The VuFind site also shows an error of “Solr index is offline.”
For good measure I recompressed the solr.war file to no avail.
I’ll drop a line in the Solr list and see if they have any feedback on known issues.
Thanks,
Russ
Simon Spero
--
Bernhardt, Russell (CIV)
Yeah, we have vufind in /opt (arbitrary choice; we liked it better than the fairly cluttered /usr/local) and solr.war is there.
I found that when used completely out of the box jetty, I would get the ZipException. But when I copied over the Vufind 1.2 jetty folder and overwrite the old 1.2 solr.war with the new 1.3 solr.war, it would start up and just give errors about missing Solr configs, which is a huge improvement. Unfortunately, the marc import still fails with the same error.
I finally registered with the solrmarc list so I can post there, so I apologize for the missing direct responses. I tried running strace on the import script, and aside from libm.so.6 not being found, I didn't see anything terribly obvious (not that the strace output is easy for me to interpret).
Running the import script as root also results in the error, so I doubt it's a permissions thing. Definitely starting to look like a system incompatibility or something... I'm not sure if the missing libm.so.6 is involved or if I missed a more glaring error in the strace output.
I found that libm.so.6 was not installed on my x64_86 server, but that it was included with the i686 version of glibc. Installing glibc.i686 did not resolve the issue.
I would try a distribution other than RHEL/CentOS, but eventually this will be going into a managed datacenter so my choices are limited to those two (well, basically just RHEL).
Any more ideas? I appreciate the time you all are putting into this, it's very strange to have it fail like this when 1.2 works so well. I did re-download the tarball thinking maybe mine was corrupt, but it has the same error. As does the SVN 1.3 branch.
Thanks again,
Sent: Tuesday, February 14, 2012 5:46 AM
Demian Katz
Are you saying that you installed your own copy of Jetty for use with VuFind 1.3? Or are you saying that you replaced VuFind 1.3’s copy of Jetty with VuFind 1.2’s? If it’s the latter, I’m very confused -- Jetty was not upgraded between 1.2 and 1.3, so the jetty folder should be completely identical. I just did a diff of the releases to be sure, and every file is exactly the same. If it’s the former, I wonder if the generic Jetty install corrupted something, though I don’t know why that would be the case….
Also, for what it’s worth, there’s definitely not an inherent incompatibility between VuFind 1.3 and RHEL – the demo server at http://vufind.org/demo is running on RHEL 5.6, so unless 5.7 introduced some major incompatibility, it seems like it should work. The demo server is 32-bit, though, so maybe it’s something to do with 64-bit configurations.
What Java version are you running?
Bernhardt, Russell (CIV)
I haven’t tried the 32-bit server yet, and it wouldn’t be the first time there were strange issues between 32 vs 64-bit systems…
I did copy the jetty folder from a fresh 1.2 directory to the 1.3 install… I did a diff on the 1.2 and the 1.3, only difference is the solr.war itself, so I’m not sure what else to say about that…
I’m running Oracle’s JRE 1.6.0_30 64-bit. I just tried their 1.6.0_31 32-bit and the error persists. For 1.2, I run OpenJDK (IcedTea6 1.9.10) 64-bit and it works just fine.
Seeing as how this is a fresh 1.3 install (aside from custom templates and config.ini), it seems likely this is a 64-bit only issue. I’ll see if I can get a 32-bit server up and just clone my install to see if that resolves it.
Simon Spero
You probably did this, but strace requires the "-f" flag to follow child processes; otherwise you just see the system calls of the script.
Bernhardt, Russell (CIV)
The 32-bit install produced the same error. However, it appears that something I did in the migration of our settings from 1.2 are causing issues in 1.3. I cleaned up our customized install of 1.3 and extracted a fresh copy without copying over any settings from 1.2 (no templates, no config.ini, nothing) and the import script works without a hitch on the 64-bit. I’m going to walk through them and see if I can pin-point the exact spot it’s failing on. I didn’t extract 1.3 directly into 1.2 (just like the upgrade instructions said not to) so I’m not sure what went wrong. I also scanned over the resulting configs and they seemed fine.
I’ll continue to experiment, but it’s evident that the problem is in my configs. No clue how that’s resulting in errors in the import, but I’ll be sure to send an update when I figure it out.
Thanks again,
Russ
From: Demian Katz [mailto:demia...@villanova.edu]
Sent: Tuesday, February 14, 2012 12:23 PM
Demian Katz
I definitely can’t think of any reason why importing old settings would break your install… unless you accidentally copied over some old .jar or other library files that caused a conflict. In any case, it’s good news that you have a working 1.3 now – hopefully if you copy settings over one at a time, you can either pinpoint or avoid the problem. Good luck!