[Dspace-tech] problems with crosswalks, export site: user passwords

[Luis Alberto Maguiña Silva]

Dear Friends:

We use DSpace 3.1, we have a problem:

We exporting entire site:

/dspace/bin/dspace packager -d -a -t AIP -e [EMAIL] -o passwords=true -i 123456789/0 /path/aip.zip

But, the system presents the error:

Disseminating DSpace SITE [ hdl=123456789/0 ] to /data/aip/aip.zip

Also disseminating all child objects (recursive mode)..

This may take a while, please check your logs for ongoing status while we process each package.

Exception: Failed to export Roles via packager (see wrapped error message for more details)

org.dspace.content.crosswalk.CrosswalkInternalException: Failed to export Roles via packager (see wrapped error message for more details)

        at org.dspace.content.crosswalk.RoleCrosswalk.disseminateElement(RoleCrosswalk.java:234)

        at org.dspace.content.packager.AbstractMETSDisseminator.crosswalkToMetsElement(AbstractMETSDisseminator.java:1359)

        at org.dspace.content.packager.AbstractMETSDisseminator.makeMdSec(AbstractMETSDisseminator.java:614)

        at org.dspace.content.packager.AbstractMETSDisseminator.addToAmdSec(AbstractMETSDisseminator.java:727)

        at org.dspace.content.packager.AbstractMETSDisseminator.addAmdSec(AbstractMETSDisseminator.java:753)

        at org.dspace.content.packager.AbstractMETSDisseminator.makeManifest(AbstractMETSDisseminator.java:839)

        at org.dspace.content.packager.AbstractMETSDisseminator.writeZipPackage(AbstractMETSDisseminator.java:311)

        at org.dspace.content.packager.AbstractMETSDisseminator.disseminate(AbstractMETSDisseminator.java:258)

        at org.dspace.content.packager.DSpaceAIPDisseminator.disseminate(DSpaceAIPDisseminator.java:160)

        at org.dspace.content.packager.AbstractPackageDisseminator.disseminateAll(AbstractPackageDisseminator.java:86)

        at org.dspace.app.packager.Packager.disseminate(Packager.java:635)

        at org.dspace.app.packager.Packager.main(Packager.java:460)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

        at java.lang.reflect.Method.invoke(Method.java:597)

        at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:183)

Caused by: org.dspace.content.packager.PackageException: java.lang.NullPointerException, Reason: java.lang.NullPointerException

        at org.dspace.content.packager.RoleDisseminator.writeToStream(RoleDisseminator.java:254)

        at org.dspace.content.packager.RoleDisseminator.disseminate(RoleDisseminator.java:105)

        at org.dspace.content.crosswalk.RoleCrosswalk.disseminateElement(RoleCrosswalk.java:202)

        ... 16 more

Caused by: java.lang.NullPointerException

        at com.ctc.wstx.sw.BaseStreamWriter.writeCharacters(BaseStreamWriter.java:498)

        at org.dspace.content.packager.RoleDisseminator.writeEPerson(RoleDisseminator.java:479)

        at org.dspace.content.packager.RoleDisseminator.writeToStream(RoleDisseminator.java:241)

        ... 18 more

 But, when I write :

/dspace/bin/dspace packager -d -a -t AIP -e [EMAIL] -i 123456789/0 /path/aip.zip

The system creates zip package, but the users don't have your passwords. 

in the dspace.cfg crosswalks DSPACE-ROLE is enable.

 org.dspace.content.crosswalk.RoleCrosswalk = DSPACE-ROLES

 org.dspace.content.packager.RoleDisseminator = DSPACE-ROLES

org.dspace.content.packager.RoleIngester = DSPACE-ROLES

aip.disseminate.techMD = PREMIS, DSPACE-ROLES

Thanks for your assistance.

Luis Maguina

Lima - Perú

--
Luis Maguiña
Linux user number 386737
Estás en tierra de Linux. En las noches tranquilas, se puede escuchar en la lejanía el ruido de las máquinas de windows reiniciándose una y otra vez

[helix84]

On Mon, Mar 4, 2013 at 11:55 PM, Luis Alberto Maguiña Silva
<lmag...@pucp.pe> wrote:
> We use DSpace 3.1, we have a problem:

> Caused by: org.dspace.content.packager.PackageException:
> java.lang.NullPointerException, Reason: java.lang.NullPointerException
> at
> org.dspace.content.packager.RoleDisseminator.writeToStream(RoleDisseminator.java:254)

Hi Luis,

you say you're using DSpace 3.1. This is the place in the source code
where the NullPointerException is reported:

https://github.com/DSpace/DSpace/blob/dspace-3.1/dspace-api/src/main/java/org/dspace/content/packager/RoleDisseminator.java#L254

But that seems wrong, there's no command on that line. Did you modify
your source code in any way? What is on line 254 in that file in your
source code?


Regards,
~~helix84

Compulsory reading: DSpace Mailing List Etiquette
https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette

[Luis Alberto Maguiña Silva]

Hi helix84:

I read the file RoleDisseminator.java, in 254 line there is no a command.

 //Only disseminate an <People> element if some people exist

    238                 if(people!=null)

    239                 {

    240                     writer.writeStartElement(EPERSONS);

    241

    242                     for (EPerson eperson : people)

    243                     {

    244                         writeEPerson(eperson, writer, emitPasswords);

    245                     }

    246

    247                     writer.writeEndElement(); // EPERSONS

    248                 }

    249

    250                 writer.writeEndElement(); // DSPACE_ROLES

    251                 writer.writeEndDocument();

    252                 writer.close();

    253             }//end if Groups or People exist

    254         }

    255         catch (Exception e)

    256         {

    257             throw new PackageException(e);

    258         }

    259     }

When I write the query in psql "select * from eperson", the system show me the users with passwords.That's ok.

Luis

2013/3/5 helix84 <hel...@centrum.sk>

[Luis Alberto Maguiña Silva]

Hi.

Thanks helix84. I have tried with DSpace 1.8 and the same results. I think that the error is in "throw new PackageException(e)", Any solution? We have 1,400 users

Luis

2013/3/5 helix84 <hel...@centrum.sk>

[Luis Alberto Maguiña Silva]

Hi, helix84.

Sorry!! In Dspace 1.8 the error is in line 254, in Dspace 3.1 the error is in line 257 : throw new PackageException(e)

Luis

2013/3/5 helix84 <hel...@centrum.sk>

[helix84]

On Tue, Mar 5, 2013 at 9:21 PM, Luis Alberto Maguiña Silva
<lmag...@pucp.pe> wrote:
> Hi, helix84.
> Sorry!! In Dspace 1.8 the error is in line 254, in Dspace 3.1 the error is
> in line 257 : throw new PackageException(e)

OK, now it makes sense:

https://github.com/DSpace/DSpace/blob/dspace-1.8.2/dspace-api/src/main/java/org/dspace/content/packager/RoleDisseminator.java#L241
https://github.com/DSpace/DSpace/blob/dspace-1.8.2/dspace-api/src/main/java/org/dspace/content/packager/RoleDisseminator.java#L479

This fails because you specified -o passwords=true, which attempts to
export the password hashes. What I think happens here is that some of
your accounts don't have passwords (external auth methods, e.g. LDAP,
don't store their passwords in DSpace) and the code doesn't handle
this case. Please, file a Jira issue about this and don't forget to
mention what authentication methods you use.

As a workaround, try doing the export without passwords (which should
work). To add the passwords, also do an SQL export + import of the
eperson table.

[Luis Alberto Maguiña Silva]

Hi helix84:

Thanks!!!!! I found 4 address email without password. I deleted them and it worked!!! 

Thanks again.

Luis

2013/3/6 helix84 <hel...@centrum.sk>

[Chris Gray]

I'm currently having a similar problem with 3.1.  The command:

./dspace packager -d -a -t AIP -e [email] -i [siteprefix]/0 /[path]/[aip].zip

fails.  Note that I am NOT using the -o passwords=true option.

What may be the complicating factor is that we use a CAS-based authentication method written for us by @mire.  They are no longer under contract to us.

We get the same error message and stack trace that Luis reported.

[Chris Gray]

I'm also having trouble locating RoleDisseminator.java on our system.  As root from the root directory "find | grep "RoleDisseminator\.java" draws a blank.

--
You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech...@googlegroups.com.
To post to this group, send email to dspac...@googlegroups.com.
Visit this group at http://groups.google.com/group/dspace-tech.
For more options, visit https://groups.google.com/d/optout.

[helix84]

Please, post the full stack trace, like Luis did.

On Sun, Oct 25, 2015 at 11:43 PM, Chris Gray <cpgr...@gmail.com> wrote:

I'm also having trouble locating RoleDisseminator.java on our system.  As root from the root directory "find | grep "RoleDisseminator\.java" draws a blank.

It's possible you either don't have the source on your system or your installation was built from per-built modules (what we call a binary dspace download). Check whether you have the dspace source directory at all:

find / -name pom.xml

[Chris Gray]

Exception: Failed to export Roles via packager (see wrapped error message for more details)
org.dspace.content.crosswalk.CrosswalkInternalException: Failed to export Roles via packager (see wrapped error message for more details)
        at org.dspace.content.crosswalk.RoleCrosswalk.disseminateElement(RoleCrosswalk.java:234)
        at org.dspace.content.packager.AbstractMETSDisseminator.crosswalkToMetsElement(AbstractMETSDisseminator.java:1359)
        at org.dspace.content.packager.AbstractMETSDisseminator.makeMdSec(AbstractMETSDisseminator.java:614)
        at org.dspace.content.packager.AbstractMETSDisseminator.addToAmdSec(AbstractMETSDisseminator.java:727)
        at org.dspace.content.packager.AbstractMETSDisseminator.addAmdSec(AbstractMETSDisseminator.java:753)
        at org.dspace.content.packager.AbstractMETSDisseminator.makeManifest(AbstractMETSDisseminator.java:839)
        at org.dspace.content.packager.AbstractMETSDisseminator.writeZipPackage(AbstractMETSDisseminator.java:311)
        at org.dspace.content.packager.AbstractMETSDisseminator.disseminate(AbstractMETSDisseminator.java:258)
        at org.dspace.content.packager.DSpaceAIPDisseminator.disseminate(DSpaceAIPDisseminator.java:160)
        at org.dspace.content.packager.AbstractPackageDisseminator.disseminateAll(AbstractPackageDisseminator.java:86)

        at org.dspace.app.packager.Packager.disseminate(Packager.java:636)

        at org.dspace.app.packager.Packager.main(Packager.java:460)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:183)
Caused by: org.dspace.content.packager.PackageException: java.lang.NullPointerException, Reason: java.lang.NullPointerException

        at org.dspace.content.packager.RoleDisseminator.writeToStream(RoleDisseminator.java:257)
        at org.dspace.content.packager.RoleDisseminator.disseminate(RoleDisseminator.java:108)

        at org.dspace.content.crosswalk.RoleCrosswalk.disseminateElement(RoleCrosswalk.java:202)
        ... 16 more
Caused by: java.lang.NullPointerException
        at com.ctc.wstx.sw.BaseStreamWriter.writeCharacters(BaseStreamWriter.java:498)

        at org.dspace.content.packager.RoleDisseminator.writeEPerson(RoleDisseminator.java:448)
        at org.dspace.content.packager.RoleDisseminator.writeToStream(RoleDisseminator.java:244)
        ... 18 more

[Chris Gray]

Yes we have a source directory with pom.xml at /home/dspace/src/pom.xml.

[helix84]

https://github.com/DSpace/DSpace/blob/dspace-3.1/dspace-api/src/main/java/org/dspace/content/packager/RoleDisseminator.java#L448

This seems to be a problem with NULLs in the email field. I assume thise occurr due to your CAS auth. OTOH, it seems package exporter can't deal with null values there.

You can try this query, but I think it will just output all your CAS users:

SELECT * FROM eperson WHERE email IS NULL;

Assuming you use autoregistration, those accounts can be simply recreated once they log in again. So it would be safe to delete them (they would lose any subscriptions, group memberships, submission rights etc).

As a workaround, you may try to set a constant value for their email before export:

UPDATE eperson SET email='XXXXX' WHERE email IS NULL;

And after import, change them back:

UPDATE eperson SET email=NULL WHERE email = 'XXXXX';

I haven't actually tested those queries, so before any changes, make sure you have DB backups and know how to restore them.

[helix84]

This problem still seems to be present in current versions of DSpace.
I filed a Jira issue here:

https://jira.duraspace.org/browse/DS-2842

[Chris Gray]

UPDATE eperson SET email='XXXXX' WHERE email IS NULL;

doesn't work since the email value must be unique.

I'm successfully running the export after running:

UPDATE eperson
SET email = 'xxxxx' || (CAST (eperson_id AS text))
WHERE email is NULL;