Re: [Dspace-tech] SWORDMETSIngester error
13 views
Skip to first unread message
Tim Donohue
Aug 25, 2015, 5:05:30 PM8/25/15
to Ingram, William A, dspac...@lists.sourceforge.net
Hi Bill,
It looks like the issue here is that, by default, DSpace's SWORD
settings expect all METS files to include EPDCX (Eprints DC XML) ,
which is the SWORD recommended metadata format, or straight DC. By
default, it doesn't expect DIM format.
So, in the SWORD settings in dspace.cfg, you'll see:
# Define the metadata type EPDCX (EPrints DC XML)
# to be handled by the SWORD crosswalk configuration
#
mets.submission.crosswalk.EPDCX = SWORD
The error is happening because SWORD is unaware of your existing "DIM"
crosswalk (as it only uses crosswalks defined via
'mets.submission.crosswalk.[mdtype]' configs). So, it is complaining
that there is no configuration for:
mets.submission.crosswalk.DIM = DIM
(NOTE: This above config may look a bit odd. But, the *first* DIM
corresponds to the MDTYPE attribute specified in your METS.xml file.
So, what this setting is actually saying is: If you encounter an
"MDTYPE=DIM" or "OTHERMDTYPE=DIM" attribute in a METS file, use the
DSpace crosswalk named "DIM" to process that metadata.)
So, it is the above setting that the METSManifest.getCrosswalk() method
is complaining is missing. See the source code
at:https://fisheye3.atlassian.com/browse/dspace/dspace/tags/dspace-1.5.2/dspace-api/src/main/java/org/dspace/content/packager/METSManifest.java?hb=true#to727
I think if you added the above 'mets.submission.crosswalk.DIM' setting
this may work (admittedly, I haven't tried it though).
- Tim
On Thursday, September 08, 2011 10:48:13 AM, Ingram, William A wrote:
> Hi,
>
> I am trying to send packages to DSpace 1.5.2 via SWORD, but they are being refused due to a crosswalk error. See below the top of the stacktrace, and a snippet of my dspace.cfg below that. The package METS.xml is attached.
>
> Thanks,
> Bill
>
>
> // --- stacktrace ---
>
> 2011-09-07 20:04:19,662 ERROR org.dspace.sword.SWORDMETSIngester @ caught exception:
> org.dspace.content.crosswalk.MetadataValidationException: Cannot process METS Manifest: No crosswalk found for MDTYPE=DIM
> at org.dspace.content.packager.METSManifest.crosswalkItem(METSManifest.java:796)
> at org.dspace.content.packager.DSpaceMETSIngester.chooseItemDmd(DSpaceMETSIngester.java:174)
> at org.dspace.content.packager.AbstractMETSIngester.ingest(AbstractMETSIngester.java:390)
> at org.dspace.sword.SWORDMETSIngester.ingest(SWORDMETSIngester.java:115)
> at org.dspace.sword.CollectionDepositor.doDeposit(CollectionDepositor.java:179)
> at org.dspace.sword.DepositManager.deposit(DepositManager.java:173)
> at org.dspace.sword.DSpaceSWORDServer.doDeposit(DSpaceSWORDServer.java:153)
> at org.purl.sword.server.DepositServlet.doPost(DepositServlet.java:336)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
> ...
>
> // --- dspace.cfg ---
>
> # Crosswalk Plugins:
> plugin.named.org.dspace.content.crosswalk.IngestionCrosswalk = \
> org.dspace.content.crosswalk.DIMIngestionCrosswalk = DIM \
> org.dspace.content.crosswalk.DIMIngestionCrosswalk = dim \
> org.dspace.content.crosswalk.OAIDCIngestionCrosswalk = DC \
> org.dspace.content.crosswalk.OAIDCIngestionCrosswalk = dc \
> org.dspace.content.crosswalk.OAIDCIngestionCrosswalk = OAI_DC \
> org.dspace.content.crosswalk.OAIDCIngestionCrosswalk = oai_dc \
> org.dspace.content.crosswalk.QDCCrosswalk = QDC \
> org.dspace.content.crosswalk.QDCCrosswalk = qdc \
> org.dspace.content.crosswalk.PREMISCrosswalk = PREMIS \
> org.dspace.content.crosswalk.OREIngestionCrosswalk = ORE \
> org.dspace.content.crosswalk.OREIngestionCrosswalk = ore \
> org.dspace.content.crosswalk.NullIngestionCrosswalk = NIL \
> org.dspace.content.crosswalk.NullIngestionCrosswalk = nil
>
> plugin.selfnamed.org.dspace.content.crosswalk.IngestionCrosswalk = \
> org.dspace.content.crosswalk.XSLTIngestionCrosswalk
>
> mets.submission.crosswalk.DIM = DIM
> sword.accept-packaging.METSDSpaceSIP.identifier = http://purl.org/net/sword-types/METSDSpaceSIP
> sword.accept-packaging.METSDSpaceSIP.q = 1.0
>
> # A comma separated list of MIME types that SWORD will accept
> sword.accepts = application/zip
>
>
>
> --
> Bill Ingram
> Research Programmer
> Scholarly Communication and Repository Services
> University of Illinois at Urbana-Champaign
>
>
>
>
> ------------------------------------------------------------------------------
> Doing More with Less: The Next Generation Virtual Desktop
> What are the key obstacles that have prevented many mid-market businesses
> from deploying virtual desktops? How do next-generation virtual desktops
> provide companies an easier-to-deploy, easier-to-manage and more affordable
> virtual desktop model.http://www.accelacomm.com/jaw/sfnl/114/51426474/
>
>
> _______________________________________________
> DSpace-tech mailing list
> DSpac...@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
It looks like the issue here is that, by default, DSpace's SWORD
settings expect all METS files to include EPDCX (Eprints DC XML) ,
which is the SWORD recommended metadata format, or straight DC. By
default, it doesn't expect DIM format.
So, in the SWORD settings in dspace.cfg, you'll see:
# Define the metadata type EPDCX (EPrints DC XML)
# to be handled by the SWORD crosswalk configuration
#
mets.submission.crosswalk.EPDCX = SWORD
The error is happening because SWORD is unaware of your existing "DIM"
crosswalk (as it only uses crosswalks defined via
'mets.submission.crosswalk.[mdtype]' configs). So, it is complaining
that there is no configuration for:
mets.submission.crosswalk.DIM = DIM
(NOTE: This above config may look a bit odd. But, the *first* DIM
corresponds to the MDTYPE attribute specified in your METS.xml file.
So, what this setting is actually saying is: If you encounter an
"MDTYPE=DIM" or "OTHERMDTYPE=DIM" attribute in a METS file, use the
DSpace crosswalk named "DIM" to process that metadata.)
So, it is the above setting that the METSManifest.getCrosswalk() method
is complaining is missing. See the source code
at:https://fisheye3.atlassian.com/browse/dspace/dspace/tags/dspace-1.5.2/dspace-api/src/main/java/org/dspace/content/packager/METSManifest.java?hb=true#to727
I think if you added the above 'mets.submission.crosswalk.DIM' setting
this may work (admittedly, I haven't tried it though).
- Tim
On Thursday, September 08, 2011 10:48:13 AM, Ingram, William A wrote:
> Hi,
>
> I am trying to send packages to DSpace 1.5.2 via SWORD, but they are being refused due to a crosswalk error. See below the top of the stacktrace, and a snippet of my dspace.cfg below that. The package METS.xml is attached.
>
> Thanks,
> Bill
>
>
> // --- stacktrace ---
>
> 2011-09-07 20:04:19,662 ERROR org.dspace.sword.SWORDMETSIngester @ caught exception:
> org.dspace.content.crosswalk.MetadataValidationException: Cannot process METS Manifest: No crosswalk found for MDTYPE=DIM
> at org.dspace.content.packager.METSManifest.crosswalkItem(METSManifest.java:796)
> at org.dspace.content.packager.DSpaceMETSIngester.chooseItemDmd(DSpaceMETSIngester.java:174)
> at org.dspace.content.packager.AbstractMETSIngester.ingest(AbstractMETSIngester.java:390)
> at org.dspace.sword.SWORDMETSIngester.ingest(SWORDMETSIngester.java:115)
> at org.dspace.sword.CollectionDepositor.doDeposit(CollectionDepositor.java:179)
> at org.dspace.sword.DepositManager.deposit(DepositManager.java:173)
> at org.dspace.sword.DSpaceSWORDServer.doDeposit(DSpaceSWORDServer.java:153)
> at org.purl.sword.server.DepositServlet.doPost(DepositServlet.java:336)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
> ...
>
> // --- dspace.cfg ---
>
> # Crosswalk Plugins:
> plugin.named.org.dspace.content.crosswalk.IngestionCrosswalk = \
> org.dspace.content.crosswalk.DIMIngestionCrosswalk = DIM \
> org.dspace.content.crosswalk.DIMIngestionCrosswalk = dim \
> org.dspace.content.crosswalk.OAIDCIngestionCrosswalk = DC \
> org.dspace.content.crosswalk.OAIDCIngestionCrosswalk = dc \
> org.dspace.content.crosswalk.OAIDCIngestionCrosswalk = OAI_DC \
> org.dspace.content.crosswalk.OAIDCIngestionCrosswalk = oai_dc \
> org.dspace.content.crosswalk.QDCCrosswalk = QDC \
> org.dspace.content.crosswalk.QDCCrosswalk = qdc \
> org.dspace.content.crosswalk.PREMISCrosswalk = PREMIS \
> org.dspace.content.crosswalk.OREIngestionCrosswalk = ORE \
> org.dspace.content.crosswalk.OREIngestionCrosswalk = ore \
> org.dspace.content.crosswalk.NullIngestionCrosswalk = NIL \
> org.dspace.content.crosswalk.NullIngestionCrosswalk = nil
>
> plugin.selfnamed.org.dspace.content.crosswalk.IngestionCrosswalk = \
> org.dspace.content.crosswalk.XSLTIngestionCrosswalk
>
> mets.submission.crosswalk.DIM = DIM
> sword.accept-packaging.METSDSpaceSIP.identifier = http://purl.org/net/sword-types/METSDSpaceSIP
> sword.accept-packaging.METSDSpaceSIP.q = 1.0
>
> # A comma separated list of MIME types that SWORD will accept
> sword.accepts = application/zip
>
>
>
> --
> Bill Ingram
> Research Programmer
> Scholarly Communication and Repository Services
> University of Illinois at Urbana-Champaign
>
>
>
>
> ------------------------------------------------------------------------------
> Doing More with Less: The Next Generation Virtual Desktop
> What are the key obstacles that have prevented many mid-market businesses
> from deploying virtual desktops? How do next-generation virtual desktops
> provide companies an easier-to-deploy, easier-to-manage and more affordable
> virtual desktop model.http://www.accelacomm.com/jaw/sfnl/114/51426474/
>
>
> _______________________________________________
> DSpace-tech mailing list
> DSpac...@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
Ingram, William A
Aug 25, 2015, 5:16:07 PM8/25/15
to Robin Taylor, dspac...@lists.sourceforge.net
Thanks to both of you. I apologize for taking so long to following up. The university spam filter is a bit overzealous.
Bill
On Sep 16, 2011, at 3:05 PM, "Robin Taylor" <robin....@ed.ac.uk> wrote:
> Hi Bill,
>
> As Tim says, the METS package appears to have its metadata coded as DIM.
> DSpace 1.5.2 didn't have a crosswalk to ingest DIM. This was added in
> either 1.6 or 1.7. You need to either recreate your package to use a
> format that DSpace 1.5 can understand (Mods or EPDCX), or copy the DIM
> crosswalk from a later version of DSpace and update you dspace.cfg to
> include the DIM crosswalk.
>
> Cheers, Robin.
>> ------------------------------------------------------------------------------
>> Why Cloud-Based Security and Archiving Make Sense
>> Osterman Research conducted this study that outlines how and why cloud
>> computing security and archiving is rapidly being adopted across the IT
>> space for its ease of implementation, lower cost, and increased
>> reliability. Learn more. http://www.accelacomm.com/jaw/sfnl/114/51425301/
Bill
On Sep 16, 2011, at 3:05 PM, "Robin Taylor" <robin....@ed.ac.uk> wrote:
> Hi Bill,
>
> As Tim says, the METS package appears to have its metadata coded as DIM.
> DSpace 1.5.2 didn't have a crosswalk to ingest DIM. This was added in
> either 1.6 or 1.7. You need to either recreate your package to use a
> format that DSpace 1.5 can understand (Mods or EPDCX), or copy the DIM
> crosswalk from a later version of DSpace and update you dspace.cfg to
> include the DIM crosswalk.
>
> Cheers, Robin.
>> Why Cloud-Based Security and Archiving Make Sense
>> Osterman Research conducted this study that outlines how and why cloud
>> computing security and archiving is rapidly being adopted across the IT
>> space for its ease of implementation, lower cost, and increased
>> reliability. Learn more. http://www.accelacomm.com/jaw/sfnl/114/51425301/
Robin Taylor
Aug 25, 2015, 5:16:13 PM8/25/15
to Tim Donohue, dspac...@lists.sourceforge.net
Hi Bill,
As Tim says, the METS package appears to have its metadata coded as DIM.
DSpace 1.5.2 didn't have a crosswalk to ingest DIM. This was added in
either 1.6 or 1.7. You need to either recreate your package to use a
format that DSpace 1.5 can understand (Mods or EPDCX), or copy the DIM
crosswalk from a later version of DSpace and update you dspace.cfg to
include the DIM crosswalk.
Cheers, Robin.
On Fri, 2011-09-09 at 19:02 +0100, Tim Donohue wrote:
As Tim says, the METS package appears to have its metadata coded as DIM.
DSpace 1.5.2 didn't have a crosswalk to ingest DIM. This was added in
either 1.6 or 1.7. You need to either recreate your package to use a
format that DSpace 1.5 can understand (Mods or EPDCX), or copy the DIM
crosswalk from a later version of DSpace and update you dspace.cfg to
include the DIM crosswalk.
Cheers, Robin.
On Fri, 2011-09-09 at 19:02 +0100, Tim Donohue wrote:
> ------------------------------------------------------------------------------
> Why Cloud-Based Security and Archiving Make Sense
> Osterman Research conducted this study that outlines how and why cloud
> computing security and archiving is rapidly being adopted across the IT
> space for its ease of implementation, lower cost, and increased
> reliability. Learn more. http://www.accelacomm.com/jaw/sfnl/114/51425301/
> Why Cloud-Based Security and Archiving Make Sense
> Osterman Research conducted this study that outlines how and why cloud
> computing security and archiving is rapidly being adopted across the IT
> space for its ease of implementation, lower cost, and increased
> reliability. Learn more. http://www.accelacomm.com/jaw/sfnl/114/51425301/
Mark Diggory
Aug 25, 2015, 5:16:19 PM8/25/15
to Ingram, William A, dspac...@lists.sourceforge.net
Bill, did you see Tim's post in the thread?
--
Mark R. Diggory
@mire - www.atmire.com
2888 Loker Avenue East - Suite 305 - Carlsbad - CA - 92010
Esperantolaan 4 - Heverlee 3001 - Belgium
Mark
On Tue, Sep 13, 2011 at 11:47 AM, Ingram, William A <wing...@illinois.edu> wrote:
Hi all,
I posted this last week, but got no response. Does anyone here know anything about METSManifest.crosswalkItem? See below.
Thanks.
BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA
Learn about the latest advances in developing for the
BlackBerry® mobile platform with sessions, labs & more.
See new tools and technologies. Register for BlackBerry® DevCon today!
http://p.sf.net/sfu/rim-devcon-copy1
_______________________________________________
DSpace-tech mailing list
DSpac...@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
Mark R. Diggory
@mire - www.atmire.com
2888 Loker Avenue East - Suite 305 - Carlsbad - CA - 92010
Esperantolaan 4 - Heverlee 3001 - Belgium
Ying Jin
Aug 25, 2015, 5:20:45 PM8/25/15
to dspace-tech@lists.sourceforge.net Tech
Hi,
We are working on upgrading dspace from 1.5.2 to 1.7.1. The process
was very smooth while I worked on my workstation. However, when I
tried to move it to our testing server, I got following error message
when Importing dspace logs to solr index.
Processing file: /server/data/dspace/logs/statlogs/dspace.log.2009-11-09
Exception: Invalid version or the data in not in 'javabin' format
java.lang.RuntimeException: Invalid version or the data in not in
'javabin' format
at
org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:99)
at
org
.apache
.solr
.client
.solrj
.impl.BinaryResponseParser.processResponse(BinaryResponseParser.java:39)
at
org
.apache
.solr
.client
.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:
477)
at
org
.apache
.solr
.client
.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:
244)
at
org
.apache
.solr
.client
.solrj
.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:105)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:64)
at
org
.dspace
.statistics.util.StatisticsImporter.load(StatisticsImporter.java:354)
at
org
.dspace
.statistics.util.StatisticsImporter.main(StatisticsImporter.java:494)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun
.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
39)
at
sun
.reflect
.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:183)
We are running it on Mac OS.
Thanks for any help,
Ying
We are working on upgrading dspace from 1.5.2 to 1.7.1. The process
was very smooth while I worked on my workstation. However, when I
tried to move it to our testing server, I got following error message
when Importing dspace logs to solr index.
Processing file: /server/data/dspace/logs/statlogs/dspace.log.2009-11-09
Exception: Invalid version or the data in not in 'javabin' format
java.lang.RuntimeException: Invalid version or the data in not in
'javabin' format
at
org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:99)
at
org
.apache
.solr
.client
.solrj
.impl.BinaryResponseParser.processResponse(BinaryResponseParser.java:39)
at
org
.apache
.solr
.client
.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:
477)
at
org
.apache
.solr
.client
.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:
244)
at
org
.apache
.solr
.client
.solrj
.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:105)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:64)
at
org
.dspace
.statistics.util.StatisticsImporter.load(StatisticsImporter.java:354)
at
org
.dspace
.statistics.util.StatisticsImporter.main(StatisticsImporter.java:494)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun
.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
39)
at
sun
.reflect
.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:183)
We are running it on Mac OS.
Thanks for any help,
Ying
Ingram, William A
Aug 25, 2015, 5:31:09 PM8/25/15
to dspac...@lists.sourceforge.net
Hi all,
I posted this last week, but got no response. Does anyone here know anything about METSManifest.crosswalkItem? See below.
Thanks.
-----Original Message-----
From: Ingram, William A
Sent: Thursday, September 08, 2011 10:48 AM
To: dspac...@lists.sourceforge.net
Subject: SWORDMETSIngester error
I posted this last week, but got no response. Does anyone here know anything about METSManifest.crosswalkItem? See below.
Thanks.
-----Original Message-----
From: Ingram, William A
Sent: Thursday, September 08, 2011 10:48 AM
To: dspac...@lists.sourceforge.net
Subject: SWORDMETSIngester error
Reply all
Reply to author
Forward
0 new messages