[Dspace-tech] A More Detailed AIP Tool Question

[Patrick Etienne]

DSpace Tech List,

In further pursuit of exporting and importing data between two dspace instances, I have a somewhat more advanced question that I'm seeking help with. I'm running into a problem where an error is being thrown, letting me know that certain epersons already exist within the instance which is the import target. I've attempted a few things such as modifying the configuration of the export instance, modifying the configuration of the import instance, and removing epersons to no avail. I'll lay out the specifics below. If anyone has ideas, they'd be most appreciated.

First I'll detail the specific action that I'm attempting to make.

I have two dspace instances that I'm trying to "merge" in a specific way. One instance is a "meta" instance of sorts - it contains the structure of several different whole instances, where each outside whole instance is a community within the meta-instance. My task has been to make certain that all communities, collections and items from the original whole instance are present within the associated meta-instance community for that source instance. This means that the parent AIP objects (my term, not sure if I'm using it 100% correctly) are different for the source and destination instances. Specifically, when working with the source instance, I'm doing a full export of the entire site (-i handle/0). But when importing the export, I'm attempting to place the entire tree at a specific community within the destination instance (-p handle/1234).

Here's the command that I'm using during the import (the export goes fine):

./dspace packager -r -a -t AIP -p 123456789/2814 -e admini...@domain.ext path-to-archive/aip.2012-01-11.zip

Now, I'll paste a copy of the specific error I'm running into:

-----------------------------------

Destination parents:

Owner: 123456789/2814
Ingesting package located at /var/www/dspace_root/src/aip/albany_dev.aip.2012-01-11/albany_dev.aip.2012-01-11.zip
Also ingesting all referenced packages (recursive mode)..
This may take a while, please check your logs for ongoing status while we process each package.
org.dspace.content.crosswalk.CrosswalkInternalException: org.dspace.content.packager.PackageException: EPerson us...@domain.ext already exists.
at org.dspace.content.crosswalk.RoleCrosswalk.ingest(RoleCrosswalk.java:338)
at org.dspace.content.crosswalk.RoleCrosswalk.ingest(RoleCrosswalk.java:257)
at org.dspace.content.packager.METSManifest.crosswalkXmd(METSManifest.java:1151)
at org.dspace.content.packager.METSManifest.crosswalkObjectOtherAdminMD(METSManifest.java:1064)
at org.dspace.content.packager.AbstractMETSIngester.ingestObject(AbstractMETSIngester.java:459)
at org.dspace.content.packager.AbstractMETSIngester.ingest(AbstractMETSIngester.java:234)
at org.dspace.content.packager.AbstractPackageIngester.ingestAll(AbstractPackageIngester.java:120)
at org.dspace.app.packager.Packager.ingest(Packager.java:515)
at org.dspace.app.packager.Packager.main(Packager.java:427)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:183)
Caused by: org.dspace.content.packager.PackageException: EPerson ashley....@asurams.edu already exists.
at org.dspace.content.packager.RoleIngester.ingestDocument(RoleIngester.java:119)
at org.dspace.content.packager.RoleIngester.ingest(RoleIngester.java:448)
at org.dspace.content.crosswalk.RoleCrosswalk.ingest(RoleCrosswalk.java:334)
... 13 more

-----------------------------------

After encountering this error, I looked around in the dspace.cfg of both the source and destination instances and had two thoughts. First was that perhaps I could disable eperson information from being exported by removing dspace.cfg references to DSPACE-ROLES within the dspace.cfg. I ended up removing several DSPACE-ROLES references (this was pretty much just the best educated guess I could make based on reading up on the associated sections of the dspace.cfg) but the import with this kind of export yielded a fairly expected kind of error:

-----------------------------------

Destination parents:

Owner: 123456789/2814
Ingesting package located at /var/www/dspace_root/src/aip/albany_dev.aip.2012-01-11/albany_dev.aip.2012-01-11.zip
Also ingesting all referenced packages (recursive mode)..
This may take a while, please check your logs for ongoing status while we process each package.
org.dspace.content.crosswalk.CrosswalkInternalException: Cannot restore Group permissions on object (type=COLLECTION, handle=ALBANYDEV/58, ID=64). The Group named 'COLLECTION_64_ADMIN' is missing from DSpace. Please restore this group using the SITE AIP, or recreate it.
at org.dspace.content.crosswalk.METSRightsCrosswalk.ingest(METSRightsCrosswalk.java:479)
at org.dspace.content.crosswalk.METSRightsCrosswalk.ingest(METSRightsCrosswalk.java:426)
at org.dspace.content.packager.METSManifest.crosswalkXmd(METSManifest.java:1151)
at org.dspace.content.packager.METSManifest.crosswalkObjectOtherAdminMD(METSManifest.java:1072)
at org.dspace.content.packager.AbstractMETSIngester.ingestObject(AbstractMETSIngester.java:459)
at org.dspace.content.packager.AbstractMETSIngester.ingest(AbstractMETSIngester.java:234)
at org.dspace.content.packager.AbstractPackageIngester.ingestAll(AbstractPackageIngester.java:120)
at org.dspace.content.packager.AbstractPackageIngester.ingestAll(AbstractPackageIngester.java:165)
at org.dspace.content.packager.AbstractPackageIngester.ingestAll(AbstractPackageIngester.java:165)
at org.dspace.content.packager.AbstractPackageIngester.ingestAll(AbstractPackageIngester.java:165)
at org.dspace.content.packager.AbstractPackageIngester.ingestAll(AbstractPackageIngester.java:165)
at org.dspace.app.packager.Packager.ingest(Packager.java:515)
at org.dspace.app.packager.Packager.main(Packager.java:427)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:183)
org.dspace.content.crosswalk.CrosswalkInternalException: Cannot restore Group permissions on object (type=COLLECTION, handle=ALBANYDEV/58, ID=64). The Group named 'COLLECTION_64_ADMIN' is missing from DSpace. Please restore this group using the SITE AIP, or recreate it.

-----------------------------------

Another thing I tried hinged on the following import property:

#mets.dspaceAIP.ingest.createSubmitter = false

I'm not sure how closely related it might be, but I enabled the directive and set it to false (even though the default is listed as being false) just to see if it had an effect. It did not.

The last thing I tried was to make a copy of the destination instance, then remove the offending epersons from the instance, then try the import again. This worked okay with the first duplicate eperson, but even though I'm an administrator for the system, I did not have the option of deleting the second eperson from the system (the checkmark box from the xmlui was greyed-out and unselectable).

I'm not sure what to try from here - outside of removing epersons via SQL and trying the import again. Even then, I'm not sure what affects there might be on my destination instance from removing the necessary epersons.

Any help or advice one could offer on this issue would most definitely be appreciated.

Many Thanks!

 - Patrick Etienne

--
Patrick K. Etienne
Systems Analyst
Georgia Institute of Technology
Library & Information Center
(404) 385-8121

[helix84]

On Thu, Jan 12, 2012 at 22:12, Patrick Etienne
<patrick...@library.gatech.edu> wrote:
> First I'll detail the specific action that I'm attempting to make.
> I have two dspace instances that I'm trying to "merge" in a specific way.
> One instance is a "meta" instance of sorts - it contains the structure of
> several different whole instances, where each outside whole instance is a
> community within the meta-instance. My task has been to make certain that
> all communities, collections and items from the original whole instance are
> present within the associated meta-instance community for that source
> instance. This means that the parent AIP objects (my term, not sure if I'm
> using it 100% correctly) are different for the source and destination
> instances. Specifically, when working with the source instance, I'm doing a
> full export of the entire site (-i handle/0). But when importing the export,
> I'm attempting to place the entire tree at a specific community within the
> destination instance (-p handle/1234).

Hello Patrick,

did you succeed in this? I'm also considering to do something similiar
so any experience with it is appreciated. Especially any caveats.

Regards,
~~helix84

[Patrick Etienne]

Tim, 

Many thanks for the response! I did see DS-1105 when it came across the list and thought that it looked like it was particularly related to what I've been experiencing. I've really been putting the AIP tool through a lot of various use cases lately. I'm focused mostly on getting a couple more specific tasks done for a project, but I hope to take some time afterward for concisely documenting my experience with the tool over the past several weeks in case it helps the community either with debugging or for others looking to get more acquainted with the tool. I just pulled up your new notes on the AIP Backup and Restore wiki page with the workarounds. I really appreciate your putting those notes together, I think it'll help a great deal. Again, hopefully sometime in the near future I may have something of my own to contribute to the efforts related to the AIP tool. Thanks again Tim!

 - Patrick E.

On Wed, Feb 1, 2012 at 2:06 PM, Tim Donohue <tdon...@duraspace.org> wrote:

Hi Patrick,

Just getting around to looking at your detailed question here.

I think, in a round-a-bout way you are actually running into a newly discovered "known issue" described here:

https://jira.duraspace.org/browse/DS-1105

I've just finished updating the latest "AIP Backup & Restore" documentation with a warning about this known issue.

Based on your description, I think you should be able to follow one of described "workarounds" to allow you to submit AIPs into your "merged" DSpace instance.

The workarounds are described both in the JIRA issue above, and in a new "warning box" at the end of this section of the AIP Backup & Restore docs:

https://wiki.duraspace.org/display/DSDOC18/AIP+Backup+and+Restore#AIPBackupandRestore-SubmittinganAIPHierarchy

Hopefully this will help out.  If I misunderstood, please let me know. But, I think your underlying problem here is that if you are just submitting a *single* Community or Collection AIP to a new DSpace, currently the AIP Backup & Restore tool cannot create missing Groups/EPeople.  Rather, the AIP Backup & Restore tool is tailored more towards only creating Groups/EPeople automatically during a *full site restore*.

- Tim

ashley....@asurams.edu <mailto:ashley.williams@asurams.edu> already

------------------------------------------------------------------------------
RSA(R) Conference 2012
Mar 27 - Feb 2
Save $400 by Jan. 27
Register now!
http://p.sf.net/sfu/rsa-sfdev2dev2



_______________________________________________
DSpace-tech mailing list
DSpac...@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

[Tim Donohue]

Hi Patrick,

Just getting around to looking at your detailed question here.

I think, in a round-a-bout way you are actually running into a newly
discovered "known issue" described here:

https://jira.duraspace.org/browse/DS-1105

I've just finished updating the latest "AIP Backup & Restore"
documentation with a warning about this known issue.

Based on your description, I think you should be able to follow one of
described "workarounds" to allow you to submit AIPs into your "merged"
DSpace instance.

The workarounds are described both in the JIRA issue above, and in a new
"warning box" at the end of this section of the AIP Backup & Restore docs:

https://wiki.duraspace.org/display/DSDOC18/AIP+Backup+and+Restore#AIPBackupandRestore-SubmittinganAIPHierarchy

Hopefully this will help out. If I misunderstood, please let me know.
But, I think your underlying problem here is that if you are just
submitting a *single* Community or Collection AIP to a new DSpace,
currently the AIP Backup & Restore tool cannot create missing
Groups/EPeople. Rather, the AIP Backup & Restore tool is tailored more
towards only creating Groups/EPeople automatically during a *full site
restore*.

- Tim

On 1/12/2012 3:12 PM, Patrick Etienne wrote:

> ashley....@asurams.edu <mailto:ashley....@asurams.edu> already

[Tim Donohue]

Patrick,

Glad to help. Definitely feel free to write up any notes/hints you
come up with, as I'm sure they'd be helpful to others. That way we can
make sure it gets back into the official documentation (as appropriate)
or just up as a "How To" document on the Wiki (as appropriate).

- Tim

On Wednesday, February 01, 2012 1:36:01 PM, Patrick Etienne wrote:
> Tim,
>
> Many thanks for the response! I did see DS-1105 when it came across
> the list and thought that it looked like it was particularly related
> to what I've been experiencing. I've really been putting the AIP tool
> through a lot of various use cases lately. I'm focused mostly on
> getting a couple more specific tasks done for a project, but I hope to
> take some time afterward for concisely documenting my experience with
> the tool over the past several weeks in case it helps the community
> either with debugging or for others looking to get more acquainted
> with the tool. I just pulled up your new notes on the AIP Backup and
> Restore wiki page with the workarounds. I really appreciate your
> putting those notes together, I think it'll help a great deal. Again,
> hopefully sometime in the near future I may have something of my own
> to contribute to the efforts related to the AIP tool. Thanks again Tim!
>
> - Patrick E.
>
> On Wed, Feb 1, 2012 at 2:06 PM, Tim Donohue <tdon...@duraspace.org
> <mailto:tdon...@duraspace.org>> wrote:
>
> Hi Patrick,
>
> Just getting around to looking at your detailed question here.
>
> I think, in a round-a-bout way you are actually running into a
> newly discovered "known issue" described here:
>

> https://jira.duraspace.org/__browse/DS-1105

> <https://jira.duraspace.org/browse/DS-1105>
>
> I've just finished updating the latest "AIP Backup & Restore"
> documentation with a warning about this known issue.
>
> Based on your description, I think you should be able to follow
> one of described "workarounds" to allow you to submit AIPs into
> your "merged" DSpace instance.
>
> The workarounds are described both in the JIRA issue above, and in
> a new "warning box" at the end of this section of the AIP Backup &
> Restore docs:
>

> https://wiki.duraspace.org/__display/DSDOC18/AIP+Backup+__and+Restore#__AIPBackupandRestore-__SubmittinganAIPHierarchy

> admini...@domain.ext path-to-archive/aip.2012-01-__11.zip

>
> Now, I'll paste a copy of the specific error I'm running into:

> ------------------------------__-----

> Destination parents:
> Owner: 123456789/2814
> Ingesting package located at

> /var/www/dspace_root/src/aip/__albany_dev.aip.2012-01-11/__albany_dev.aip.2012-01-11.zip

> Also ingesting all referenced packages (recursive mode)..
> This may take a while, please check your logs for ongoing
> status while
> we process each package.

> org.dspace.content.crosswalk.__CrosswalkInternalException:
> org.dspace.content.packager.__PackageException: EPerson
> us...@domain.ext
> already exists.
> at
> org.dspace.content.crosswalk.__RoleCrosswalk.ingest(__RoleCrosswalk.java:338)
> at
> org.dspace.content.crosswalk.__RoleCrosswalk.ingest(__RoleCrosswalk.java:257)
> at
> org.dspace.content.packager.__METSManifest.crosswalkXmd(__METSManifest.java:1151)
> at
> org.dspace.content.packager.__METSManifest.__crosswalkObjectOtherAdminMD(__METSManifest.java:1064)
> at
> org.dspace.content.packager.__AbstractMETSIngester.__ingestObject(__AbstractMETSIngester.java:459)
> at
> org.dspace.content.packager.__AbstractMETSIngester.ingest(__AbstractMETSIngester.java:234)
> at
> org.dspace.content.packager.__AbstractPackageIngester.__ingestAll(__AbstractPackageIngester.java:__120)
> at org.dspace.app.packager.__Packager.ingest(Packager.java:__515)
> at org.dspace.app.packager.__Packager.main(Packager.java:__427)
> at sun.reflect.__NativeMethodAccessorImpl.__invoke0(Native Method)
> at
> sun.reflect.__NativeMethodAccessorImpl.__invoke(__NativeMethodAccessorImpl.java:__39)
> at
> sun.reflect.__DelegatingMethodAccessorImpl.__invoke(__DelegatingMethodAccessorImpl.__java:25)
> at java.lang.reflect.Method.__invoke(Method.java:597)
> at
> org.dspace.app.launcher.__ScriptLauncher.main(__ScriptLauncher.java:183)
> Caused by: org.dspace.content.packager.__PackageException: EPerson
> ashley....@asurams.edu
> <mailto:ashley....@asurams.edu>
> <mailto:ashley.williams@__asurams.edu

> <mailto:ashley....@asurams.edu>> already
>
> exists.
> at

> org.dspace.content.packager.__RoleIngester.ingestDocument(__RoleIngester.java:119)
> at
> org.dspace.content.packager.__RoleIngester.ingest(__RoleIngester.java:448)
> at
> org.dspace.content.crosswalk.__RoleCrosswalk.ingest(__RoleCrosswalk.java:334)
> ... 13 more
> ------------------------------__-----

>
> After encountering this error, I looked around in the
> dspace.cfg of both
> the source and destination instances and had two thoughts.
> First was
> that perhaps I could disable eperson information from being
> exported by
> removing dspace.cfg references to DSPACE-ROLES within the
> dspace.cfg. I
> ended up removing several DSPACE-ROLES references (this was
> pretty much
> just the best educated guess I could make based on reading up
> on the
> associated sections of the dspace.cfg) but the import with
> this kind of
> export yielded a fairly expected kind of error:
>

> ------------------------------__-----

> Destination parents:
> Owner: 123456789/2814
> Ingesting package located at

> /var/www/dspace_root/src/aip/__albany_dev.aip.2012-01-11/__albany_dev.aip.2012-01-11.zip

> Also ingesting all referenced packages (recursive mode)..
> This may take a while, please check your logs for ongoing
> status while
> we process each package.

> org.dspace.content.crosswalk.__CrosswalkInternalException:

> Cannot restore
> Group permissions on object (type=COLLECTION, handle=ALBANYDEV/58,
> ID=64). The Group named 'COLLECTION_64_ADMIN' is missing from
> DSpace.
> Please restore this group using the SITE AIP, or recreate it.
> at

> org.dspace.content.crosswalk.__METSRightsCrosswalk.ingest(__METSRightsCrosswalk.java:479)
> at
> org.dspace.content.crosswalk.__METSRightsCrosswalk.ingest(__METSRightsCrosswalk.java:426)
> at
> org.dspace.content.packager.__METSManifest.crosswalkXmd(__METSManifest.java:1151)
> at
> org.dspace.content.packager.__METSManifest.__crosswalkObjectOtherAdminMD(__METSManifest.java:1072)
> at
> org.dspace.content.packager.__AbstractMETSIngester.__ingestObject(__AbstractMETSIngester.java:459)
> at
> org.dspace.content.packager.__AbstractMETSIngester.ingest(__AbstractMETSIngester.java:234)
> at
> org.dspace.content.packager.__AbstractPackageIngester.__ingestAll(__AbstractPackageIngester.java:__120)
> at
> org.dspace.content.packager.__AbstractPackageIngester.__ingestAll(__AbstractPackageIngester.java:__165)
> at
> org.dspace.content.packager.__AbstractPackageIngester.__ingestAll(__AbstractPackageIngester.java:__165)
> at
> org.dspace.content.packager.__AbstractPackageIngester.__ingestAll(__AbstractPackageIngester.java:__165)
> at
> org.dspace.content.packager.__AbstractPackageIngester.__ingestAll(__AbstractPackageIngester.java:__165)
> at org.dspace.app.packager.__Packager.ingest(Packager.java:__515)
> at org.dspace.app.packager.__Packager.main(Packager.java:__427)
> at sun.reflect.__NativeMethodAccessorImpl.__invoke0(Native Method)
> at
> sun.reflect.__NativeMethodAccessorImpl.__invoke(__NativeMethodAccessorImpl.java:__39)
> at
> sun.reflect.__DelegatingMethodAccessorImpl.__invoke(__DelegatingMethodAccessorImpl.__java:25)
> at java.lang.reflect.Method.__invoke(Method.java:597)
> at
> org.dspace.app.launcher.__ScriptLauncher.main(__ScriptLauncher.java:183)
> org.dspace.content.crosswalk.__CrosswalkInternalException:

> Cannot restore
> Group permissions on object (type=COLLECTION, handle=ALBANYDEV/58,
> ID=64). The Group named 'COLLECTION_64_ADMIN' is missing from
> DSpace.
> Please restore this group using the SITE AIP, or recreate it.

> ------------------------------__-----

>
> Another thing I tried hinged on the following import property:

> #mets.dspaceAIP.ingest.__createSubmitter = false

> (404) 385-8121 <tel:%28404%29%20385-8121>
>
>
>
>
> ------------------------------__------------------------------__------------------

> RSA(R) Conference 2012
> Mar 27 - Feb 2
> Save $400 by Jan. 27
> Register now!

> http://p.sf.net/sfu/rsa-__sfdev2dev2
> <http://p.sf.net/sfu/rsa-sfdev2dev2>
>
>
>
> _________________________________________________
> DSpace-tech mailing list
> DSpac...@lists.sourceforge.__net
> <mailto:DSpac...@lists.sourceforge.net>
> https://lists.sourceforge.net/__lists/listinfo/dspace-tech