ERROR oai import

[Freddy Guerrero]

Hi, I use dspace v. 6.3, jspui, and when executing "dspace oai import ", I get this error:

java.lang.NullPointerException
        at org.dspace.storage.bitstore.BitstreamStorageServiceImpl.retrieve(BitstreamStorageServiceImpl.java:216)
        at org.dspace.content.BitstreamServiceImpl.retrieve(BitstreamServiceImpl.java:309)
        at org.dspace.xoai.util.ItemUtils.retrieveMetadata(ItemUtils.java:298)
        at org.dspace.xoai.app.XOAI.index(XOAI.java:421)
        at org.dspace.xoai.app.XOAI.index(XOAI.java:280)
        at org.dspace.xoai.app.XOAI.index(XOAI.java:179)
        at org.dspace.xoai.app.XOAI.index(XOAI.java:145)
        at org.dspace.xoai.app.XOAI.main(XOAI.java:560)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.dspace.app.launcher.ScriptLauncher.runOneCommand(ScriptLauncher.java:229)
        at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:81)

[Tim Donohue]

Hi Freddy,

That error seems to be saying that one of your Item's License files is missing. Here's the OAI code that is throwing the NullPointerException (after the License bitstream cannot be retrieved):
https://github.com/DSpace/DSpace/blob/dspace-6_x/dspace-oai/src/main/java/org/dspace/xoai/util/ItemUtils.java#L298

OAI import attempts to add the content of any License files to a "license" field (for some metadata formats).  Here's an example from the demo site, where you can see a "DSpaceDepositLicense" when you ask for "metadataPrefix=mets": http://demo.dspace.org/oai/request?verb=GetRecord&metadataPrefix=mets&identifier=oai:demo.dspace.org:10673/5 

In any case, it sounds like one of your Item's has an associated Bitstream object (that represents a License), but that Bitstream's content is missing from your filesystem.  I'm not sure if there's an easy way to track down the exact item, but you might look more closely at the log files *just before* this error occurs to see if OAI is logging exactly which Item it is attempting to import when this happens.

Good luck, and let us know on this list if you have further questions.

Tim

--
All messages to this mailing list should adhere to the DuraSpace Code of Conduct: https://duraspace.org/about/policies/code-of-conduct/
---
You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech...@googlegroups.com.
To post to this group, send email to dspac...@googlegroups.com.
Visit this group at https://groups.google.com/group/dspace-tech.
For more options, visit https://groups.google.com/d/optout.

--

Tim Donohue
Technical Lead for DSpace & DSpaceDirect
DuraSpace.org | DSpace.org | DSpaceDirect.org

[Freddy Guerrero]

Thanks Tim for your answer, can someone provide us with a recommendation on how to identify the records that do not contain the license files?

Regards

[Claudia Jürgen]

Hello Freddy,

you can query the database:

select * from item where in_archive=true and uuid not in (select
item_id from item2bundle join metadatavalue on
item2bundle.bundle_id=metadatavalue.dspace_object_id where
metadata_field_id in (select metadata_field_id from
metadatafieldregistry where metadata_schema_id=1 and element='title' and
qualifier is null) and text_value='LICENSE');

Did you ever import items, most likely these had no license specified.

Hope this helps

Claudia Jürgen

--
Claudia Juergen
Eldorado

Technische Universität Dortmund
Universitätsbibliothek
Vogelpothsweg 76
44227 Dortmund

Tel.: +49 231-755 40 43
Fax: +49 231-755 40 32
claudia...@tu-dortmund.de
www.ub.tu-dortmund.de

Wichtiger Hinweis: Die Information in dieser E-Mail ist vertraulich. Sie ist ausschließlich für den Adressaten bestimmt. Sollten Sie nicht der für diese E-Mail bestimmte Adressat sein, unterrichten Sie bitte den Absender und vernichten Sie diese Mail. Vielen Dank.
Unbeschadet der Korrespondenz per E-Mail, sind unsere Erklärungen ausschließlich final rechtsverbindlich, wenn sie in herkömmlicher Schriftform (mit eigenhändiger Unterschrift) oder durch Übermittlung eines solchen Schriftstücks per Telefax erfolgen.

Important note: The information included in this e-mail is confidential. It is solely intended for the recipient. If you are not the intended recipient of this e-mail please contact the sender and delete this message. Thank you. Without prejudice of e-mail correspondence, our statements are only legally binding when they are made in the conventional written form (with personal signature) or when such documents are sent by fax.

[Freddy Guerrero]

Hello Claudia

Thanks for your help, run the script and I left the items that were not licensed, then one by one I added the license, and actually the license file was generated. I run the script again and the same items keep coming up, should I run something else?

Regards,

Freddy

[Claudia Jürgen]

Hello Freddy,

did you recreate the oai index after having corrected the items?
This is done with
[dspace]/bin/dspace oai -c import

Hope this helps

Claudia Jürgen

>> claudia...@tu-dortmund.de <javascript:>

[Freddy Guerrero]

Hi Claudia

Yes, execute:

$DSPACE/bin/dspace oai clean-cache

$DSPACE/bin/dspace oai import -c

i have error:

java.lang.NullPointerException
        at org.dspace.storage.bitstore.BitstreamStorageServiceImpl.retrieve(BitstreamStorageServiceImpl.java:216)
        at org.dspace.content.BitstreamServiceImpl.retrieve(BitstreamServiceImpl.java:309)

        at org.dspace.xoai.util.ItemUtils.retrieveMetadata(ItemUtils.java:298)

        at org.dspace.xoai.app.XOAI.index(XOAI.java:421)
        at org.dspace.xoai.app.XOAI.index(XOAI.java:280)
        at org.dspace.xoai.app.XOAI.index(XOAI.java:179)
        at org.dspace.xoai.app.XOAI.index(XOAI.java:145)
        at org.dspace.xoai.app.XOAI.main(XOAI.java:560)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.dspace.app.launcher.ScriptLauncher.runOneCommand(ScriptLauncher.java:229)
        at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:81)

[Claudia Jürgen]

Hi Freddy,

that is the same error as before, see Tim's answer from the 28th of
February for details.
Track down the item in question and correct it before creating the oai
index.

cu

Claudia Jürgen

[Freddy Guerrero]

Hi friends, I continue looking for the solution to this problem without achieving it, I have performed the indicated actions, when executing the command: 

[dspace]/bin/dspace oai-import -c

[dspace]/bin/dspace oai-import -v

I found this error:

Item 2cf599c6-e004-45d3-8710-bb01be6506fa with handle 10469/4816 indexed

java.lang.NullPointerException

        at org.dspace.storage.bitstore.BitstreamStorageServiceImpl.retrieve(BitstreamStorageServiceImpl.java:216)

        at org.dspace.content.BitstreamServiceImpl.retrieve(BitstreamServiceImpl.java:309)

        at org.dspace.xoai.util.ItemUtils.retrieveMetadata(ItemUtils.java:298)

        at org.dspace.xoai.app.XOAI.index(XOAI.java:421)

        at org.dspace.xoai.app.XOAI.index(XOAI.java:280)

        at org.dspace.xoai.app.XOAI.indexAll(XOAI.java:227)

        at org.dspace.xoai.app.XOAI.index(XOAI.java:143)

        at org.dspace.xoai.app.XOAI.main(XOAI.java:560)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.dspace.app.launcher.ScriptLauncher.runOneCommand(ScriptLauncher.java:229)

        at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:81)

The record mentioned if you have a license:

  






El martes, 26 de febrero de 2019, 0:04:39 (UTC-5), Freddy Guerrero escribió:

[Freddy Guerrero]

Dear friends please request your help, we have done manual debugging of the records adding the respective license, but the error continues:

My local.cfg file:

assetstore.dir = /dspace6/assetstore

assetstore.dir.1 = /dspace6/assetstore2

/dspace6/config/spring/api/bitstore.xml:

<?xml version="1.0" encoding="UTF-8"?>

<beans xmlns="http://www.springframework.org/schema/beans"

       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd" default-lazy-init="true">

    <bean name="org.dspace.storage.bitstore.BitstreamStorageService" class="org.dspace.storage.bitstore.BitstreamStorageServiceImpl">

        <property name="incoming" value="0"/>

        <property name="stores">

            <map>

                <entry key="0" value-ref="localStore"/>

                <!--<entry key="1" value-ref="s3Store"/>-->

            </map>

        </property>

    </bean>

    <bean name="localStore" class="org.dspace.storage.bitstore.DSBitStoreService" scope="singleton">

        <property name="baseDir" value="${assetstore.dir}"/>

    </bean>

    <bean name="s3Store" class="org.dspace.storage.bitstore.S3BitStoreService" scope="singleton">

        <!-- AWS Security credentials, with policies for specified bucket -->

        <property name="awsAccessKey" value=""/>

        <property name="awsSecretKey" value=""/>

        <!-- S3 bucket name to store assets in. example: longsight-dspace-auk -->

        <property name="bucketName" value=""/>

        <!-- AWS S3 Region to use: {us-east-1, us-west-1, eu-west-1, eu-central-1, ap-southeast-1, ... } -->

        <!-- Optional, sdk default is us-east-1 -->

        <property name="awsRegionName" value=""/>

        <!-- Subfolder to organize assets within the bucket, in case this bucket is shared  -->

        <!-- Optional, default is root level of bucket -->

        <property name="subfolder" value=""/>

    </bean>

<bean name="localStore2" class="org.dspace.storage.bitstore.DSBitStoreService" scope="singleton">

        <property name="baseDir" value="${assetstore.dir.1}"/>

    </bean>

    <bean name="s3Store2" class="org.dspace.storage.bitstore.S3BitStoreService" scope="singleton">

        <!-- AWS Security credentials, with policies for specified bucket -->

        <property name="awsAccessKey" value=""/>

        <property name="awsSecretKey" value=""/>

        <!-- S3 bucket name to store assets in. example: longsight-dspace-auk -->

        <property name="bucketName" value=""/>

        <!-- AWS S3 Region to use: {us-east-1, us-west-1, eu-west-1, eu-central-1, ap-southeast-1, ... } -->

        <!-- Optional, sdk default is us-east-1 -->

        <property name="awsRegionName" value=""/>

        <!-- Subfolder to organize assets within the bucket, in case this bucket is shared  -->

        <!-- Optional, default is root level of bucket -->

        <property name="subfolder" value=""/>

    </bean>

    <!-- <bean name="localStore2 ... -->

    <!-- <bean name="s3Store2 ... -->

</beans>

El martes, 26 de febrero de 2019, 0:04:39 (UTC-5), Freddy Guerrero escribió: