Dspace 7.2 - import of data from 5.8 has taken over a month

[Romana Challans]

Hi all,

Inherited a system that is old (5.8), on an EOL box, and is having real issues post Log4j and some necessary network changes. Finally got permission to do a new version, so we are importing our Library collection into 7.2. We have 36G in our assetstore.

/opt/dspace/bin/dspace packager -r -a -k -t AIP -o skipIfParentMissing=true -e <ADMIN_EMAIL>-i <HANDLE>/0 /opt/AIP/fac_site.zip

Current Meta-data Entry Count is 1530880

However, this process has been running for about seven weeks now!

1. Is this normal?

2. Is there any way we can see how much longer it will take? (management is nervous, understandably, as the current live version is very fragile)

3. Is there anyway to expedite this?

Thanks very much for any assistance that can be offered.

   

[Tim Donohue]

Hi,

No that is definitely not normal behavior, especially for a relatively small collection (36GB isn't that large).  I would highly recommend checking the dspace.log (log files) to see if you are receiving errors or warnings.  If you are seeing no activity in the logs then the process has likely stopped completely and nothing is actively changing.

As a sidenote, it's also worth noting that there's no need to do an export (to AIPs) and reimport (from AIPs) when moving from 5.8 to 7.2.   The easier way to do this upgrade is to simply follow the upgrade instructions: https://wiki.lyrasis.org/display/DSDOC7x/Upgrading+DSpace   If the 5.8 database is on a different machine, it's still possible to do this same sort of upgrade, but you'd need a few extra steps:
1. First, it may be easiest to install DSpace 7 without any data first (just to make sure everything works well).  Once satisfied, stop Tomcat (backend)

2. Copy your entire database contents to your new database (e.g. for Postgres, use tools like pg_dump and pg_restore). This copies all metadata/relationships over to the new server

3. Copy over your entire [dspace]/assetstore folder (including all subdirectories) to the new location. This copies all files over to the new server.

4. Run "./dspace database migrate ignored" to upgrade your database/data up to 7.x compatibility

5. Start Tomcat (backend) & frontend

6. Finally, reindex all content into the search engine (Run "./dspace index-discovery -b")

It's really hard to give you better advice as it's unclear what could be going on based on what you've shared.  But, this doesn't sound like normal behavior.

If you find anything else in the logs that are clues, let us know on this list.

Tim

[Romana Challans]

Tim,

I cannot thank you enough. That was the solution! Weeks of tearing our hair out...

We are wiser (if less hirsute) now, thank you:)

Romana