Migration from 5.4 to 7.6

Andrew K

unread,

Sep 30, 2023, 11:03:24 AM9/30/23

to DSpace Technical Support

Hello!

I am trying to upgrade from 5.4 to the latest 7.6 version.

I installed the new server and copied the assetstore and database.

Then did

sudo /dspace/bin/dspace database migrate ignored

sudo /dspace/bin/dspace database info

Database Type: postgres
Database URL: jdbc:postgresql://localhost:5432/dspace
Database Schema: public
Database Username: dspace
Database Software: PostgreSQL version 12.16 (Ubuntu 12.16-0ubuntu0.20.04.1)
Database Driver: PostgreSQL JDBC Driver version 42.6.0
PostgreSQL 'pgcrypto' extension installed/up-to-date? true (version=1.3)
FlywayDB Version: 8.4.4

+-----------+----------------+------------------------------------------------------------------+----------+---------------------+----------+
| Category | Version | Description | Type | Installed On | State |
+-----------+----------------+------------------------------------------------------------------+----------+---------------------+----------+
| | 1 | << Flyway Baseline >> | BASELINE | 2023-09-30 17:31:15 | Baseline |
| Versioned | 1.1 | Initial DSpace 1.1 database schema | SQL | 2023-09-30 17:31:15 | Success |
| Versioned | 1.2 | Upgrade to DSpace 1.2 schema | SQL | 2023-09-30 17:31:15 | Success |
| Versioned | 1.3 | Upgrade to DSpace 1.3 schema | SQL | 2023-09-30 17:31:15 | Success |
| Versioned | 1.3.9 | Drop constraint for DSpace 1 4 schema | JDBC | 2023-09-30 17:31:16 | Success |
| Versioned | 1.4 | Upgrade to DSpace 1.4 schema | SQL | 2023-09-30 17:31:16 | Success |
| Versioned | 1.5 | Upgrade to DSpace 1.5 schema | SQL | 2023-09-30 17:31:16 | Success |
| Versioned | 1.5.9 | Drop constraint for DSpace 1 6 schema | JDBC | 2023-09-30 17:31:16 | Success |
| Versioned | 1.6 | Upgrade to DSpace 1.6 schema | SQL | 2023-09-30 17:31:16 | Success |
| Versioned | 1.7 | Upgrade to DSpace 1.7 schema | SQL | 2023-09-30 17:31:16 | Success |
| Versioned | 1.8 | Upgrade to DSpace 1.8 schema | SQL | 2023-09-30 17:31:16 | Success |
| Versioned | 3.0 | Upgrade to DSpace 3.x schema | SQL | 2023-09-30 17:31:16 | Success |
| Versioned | 4.0 | Upgrade to DSpace 4.x schema | SQL | 2023-09-30 17:31:16 | Success |
| Versioned | 4.9.2015.10.26 | DS-2818 registry update | SQL | 2023-09-30 17:31:16 | Success |
| Versioned | 5.0.2014.08.08 | DS-1945 Helpdesk Request a Copy | SQL | 2023-09-30 17:31:16 | Success |
| Versioned | 5.0.2014.09.25 | DS 1582 Metadata For All Objects drop constraint | JDBC | 2023-09-30 17:31:16 | Success |
| Versioned | 5.0.2014.09.26 | DS-1582 Metadata For All Objects | SQL | 2023-09-30 17:31:16 | Success |
| Versioned | 5.0.2014.11.04 | Enable XMLWorkflow Migration | JDBC | 2023-09-30 17:31:16 | Success |
| Versioned | 5.6.2016.08.23 | DS-3097 | SQL | 2023-09-30 17:31:17 | Success |
| Versioned | 5.7.2017.04.11 | DS-3563 Index metadatavalue resource type id column | SQL | 2023-09-30 17:31:17 | Success |
| Versioned | 5.7.2017.05.05 | DS 3431 Add Policies for BasicWorkflow | JDBC | 2023-09-30 17:31:17 | Success |
| Versioned | 6.0.2015.03.06 | DS 2701 Dso Uuid Migration | JDBC | 2023-09-30 17:31:17 | Success |
| Versioned | 6.0.2015.03.07 | DS-2701 Hibernate migration | SQL | 2023-09-30 17:31:17 | Success |
| Versioned | 6.0.2015.08.31 | DS 2701 Hibernate Workflow Migration | JDBC | 2023-09-30 17:31:18 | Success |
| Versioned | 6.0.2015.09.01 | DS 2701 Enable XMLWorkflow Migration | JDBC | 2023-09-30 17:31:18 | Success |
| Versioned | 6.0.2016.01.03 | DS-3024 | SQL | 2023-09-30 17:31:18 | Success |
| Versioned | 6.0.2016.01.26 | DS 2188 Remove DBMS Browse Tables | JDBC | 2023-09-30 17:31:18 | Success |
| Versioned | 6.0.2016.02.25 | DS-3004-slow-searching-as-admin | SQL | 2023-09-30 17:31:18 | Success |
| Versioned | 6.0.2016.04.01 | DS-1955 Increase embargo reason | SQL | 2023-09-30 17:31:18 | Success |
| Versioned | 6.0.2016.04.04 | DS-3086-OAI-Performance-fix | SQL | 2023-09-30 17:31:18 | Success |
| Versioned | 6.0.2016.04.14 | DS-3125-fix-bundle-bitstream-delete-rights | SQL | 2023-09-30 17:31:18 | Success |
| Versioned | 6.0.2016.05.10 | DS-3168-fix-requestitem item id column | SQL | 2023-09-30 17:31:18 | Success |
| Versioned | 6.0.2016.07.21 | DS-2775 | SQL | 2023-09-30 17:31:18 | Success |
| Versioned | 6.0.2016.07.26 | DS-3277 fix handle assignment | SQL | 2023-09-30 17:31:18 | Success |
| Versioned | 6.0.2016.08.23 | DS-3097 | SQL | 2023-09-30 17:31:18 | Success |
| Versioned | 6.1.2017.01.03 | DS 3431 Add Policies for BasicWorkflow | JDBC | 2023-09-30 17:31:18 | Success |
| Versioned | 7.0.2017.10.12 | DS-3542-stateless-sessions | SQL | 2023-09-30 17:31:18 | Success |
| Versioned | 7.0.2018.04.03 | Upgrade Workflow Policy | JDBC | 2023-09-30 17:31:18 | Success |
| Versioned | 7.0.2018.04.16 | dspace-entities | SQL | 2023-09-30 17:31:18 | Success |
| Versioned | 7.0.2018.06.07 | DS-3851-permission | SQL | 2023-09-30 17:31:18 | Success |
| Versioned | 7.0.2019.05.02 | DS-4239-workflow-xml-migration | SQL | 2023-09-30 17:31:18 | Success |
| Versioned | 7.0.2019.06.14 | scripts-and-process | SQL | 2023-09-30 17:31:18 | Success |
| Versioned | 7.0.2019.07.31 | Retrieval of name variant | SQL | 2023-09-30 17:31:18 | Success |
| Versioned | 7.0.2019.11.13 | relationship type copy left right | SQL | 2023-09-30 17:31:18 | Success |
| Versioned | 7.0.2020.01.08 | DS-626-statistics-tracker | SQL | 2023-09-30 17:31:18 | Success |
| Versioned | 7.0.2020.10.31 | CollectionCommunity Metadata Handle | JDBC | 2023-09-30 17:31:18 | Success |
| Versioned | 7.0.2021.01.22 | Remove basic workflow | SQL | 2023-09-30 17:31:18 | Success |
| Versioned | 7.0.2021.02.08 | tilted rels | SQL | 2023-09-30 17:31:18 | Success |
| Versioned | 7.0.2021.03.18 | Move entity type to dspace schema | SQL | 2023-09-30 17:31:18 | Success |
| Versioned | 7.0.2021.09.24 | Move entity type from item template to collection | SQL | 2023-09-30 17:31:18 | Success |
| Versioned | 7.0.2021.10.04 | alter collection table drop workflow stem columns | SQL | 2023-09-30 17:31:18 | Success |
| Versioned | 7.1.2021.10.18 | Fix MDV place after migrating from DSpace 5 | SQL | 2023-09-30 17:31:18 | Success |
| Versioned | 7.3.2022.04.29 | orcid queue and history | SQL | 2023-09-30 17:31:18 | Success |
| Versioned | 7.3.2022.05.16 | Orcid token table | SQL | 2023-09-30 17:31:18 | Success |
| Versioned | 7.3.2022.06.16 | process to group | SQL | 2023-09-30 17:31:18 | Success |
| Versioned | 7.3.2022.06.20 | add last version status column to relationship table | SQL | 2023-09-30 17:31:18 | Success |
| Versioned | 7.5.2022.12.01 | add table subscriptionparamter change columns subscription table | SQL | 2023-09-30 17:31:18 | Success |
| Versioned | 7.5.2022.12.06 | index action resource policy | SQL | 2023-09-30 17:31:18 | Success |
| Versioned | 7.5.2022.12.09 | Supervision Orders table | SQL | 2023-09-30 17:31:18 | Success |
| Versioned | 7.5.2022.12.15 | system wide alerts | SQL | 2023-09-30 17:31:18 | Success |
| Versioned | 7.6.2023.03.17 | Remove unused sequence | SQL | 2023-09-30 17:31:18 | Success |
| Versioned | 7.6.2023.03.24 | Update PNG in bitstream format registry | SQL | 2023-09-30 17:31:18 | Success |
| Versioned | 7.6.2023.03.29 | orcid queue and history descriptions to text type | SQL | 2023-09-30 17:31:18 | Success |
| Versioned | 7.6.2023.04.19 | process parameters to text type | SQL | 2023-09-30 17:31:18 | Success |
+-----------+----------------+------------------------------------------------------------------+----------+---------------------+----------+

Everything looks OK.

Then I do

sudo /dspace/bin/dspace index-discovery -b
The script has started
(Re)building index from scratch.
Done with indexing
The script has completed

sudo /dspace/bin/dspace filter-media
The script has started
The script has completed

But both scripts run very fast and the server shows no content (the interface is available).

What can be wrong?

Thanks,

Andrew

DSpace Technical Support

unread,

Oct 4, 2023, 12:42:10 PM10/4/23

to DSpace Technical Support

Hi Andrew,

Based on the information you shared, it appears (to me) that you ran the "dspace database migrate ignored" command on an **empty** database. If you look, the "info" command reported that the "Installed On" date for every single migration was Sept 30, 2023. That implies that the database had no prior content or database tables.

I suspect you may have forgotten to migrate your database over from 5.x to 7.x? If it's helpful, we have a migration guide which can walk through those steps here: https://wiki.lyrasis.org/display/DSDOC7x/Migrating+DSpace+to+a+new+server

It is also possible to simply follow the Upgrading guide, but it tends to "assume" your database data is already in place: https://wiki.lyrasis.org/display/DSDOC7x/Upgrading+DSpace

Tim

Andrew K

unread,

Oct 4, 2023, 3:41:35 PM10/4/23

to DSpace Technical Support

Hello Tim,

Thank you! I don't know how but the database was not imported. The problem solved.

Another problem: on the old server I run

root@dspace:/www/dspace/bin # ./dspace solr-export-statistics
Exception: Expected mime type application/octet-stream but got text/html. <!DOCTYPE html><html><head><title>Apache Tomcat/8.0.28 - Error report</title><style type="text/css">H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}.line {height: 1px; background-color: #525D76; border: none;}</style> </head><body><h1>HTTP Status 500 - {msg=SolrCore 'statistics' is not available due to init failure: Error opening new searcher,trace=org.apache.solr.common.SolrException: SolrCore 'statistics' is not available due to init failure: Error opening new searcher
at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:745)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:299)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.dspace.solr.filters.LocalHostRestrictionFilter.doFilter(LocalHostRestrictionFilter.java:50)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)

and so on, very long error message.

The old server is running.

What is the problem, how can I move the solr data to the new server?

середа, 4 жовтня 2023 р. о 19:42:10 UTC+3 DSpace Technical Support пише:

Andrew K

unread,

Oct 4, 2023, 3:43:58 PM10/4/23

to DSpace Technical Support

The error message also has this string

Caused by: java.nio.file.NoSuchFileException: /www/dspace/solr/statistics/data/index/segments_1

середа, 4 жовтня 2023 р. о 22:41:35 UTC+3 Andrew K пише:

DSpace Technical Support

unread,

Oct 4, 2023, 4:08:42 PM10/4/23

to DSpace Technical Support

Hi Andrew,

As that "Caused by" error notes, it appears that Solr cannot find your statistics data in the "/www/dspace/solr/statistics/data/" folder (and the subfolders under that).

Are you certain you have statistics data in that specific location on your filesystem? Are you sure that Solr is up and running while you try to do the export?

You also might want to check Solr's own logs, as it's possible more detailed information is available there. It could be a simple permissions issue, or a configuration issue, or any number of other things.

Good luck,

Tim

Andrew K

unread,

Oct 6, 2023, 3:10:13 PM10/6/23

to DSpace Technical Support

Thank you, Tim!

It looks like solr is not installed on that server. And it has been online for 7 years like this. Pity.

Another question.

I moved the database and assetstore to the new server.

Then I did

sudo /dspace/bin/dspace filter-media

It took more than a day to complete (12+K items).

Now I try

sudo /dspace/bin/dspace index-discovery

but it fails:

The script has started
Updating Index
java.lang.IllegalArgumentException: More than one result found
at org.dspace.core.AbstractHibernateDAO.uniqueResult(AbstractHibernateDAO.java:242)
at org.dspace.content.dao.impl.WorkspaceItemDAOImpl.findByItem(WorkspaceItemDAOImpl.java:81)

What is going on, please?

Also, when I imported the database, there was an error in the end

...
ALTER TABLE
REVOKE
psql:db.sql:603904: ERROR: role "pgsql" does not exist
psql:db.sql:603905: ERROR: role "pgsql" does not exist
GRANT

There was psql role and user instead of postgres on the old server.

Is that a problem I should correct somehow?

Thanks,

Andrew

середа, 4 жовтня 2023 р. о 23:08:42 UTC+3 DSpace Technical Support пише:

Andrew K

unread,

Oct 6, 2023, 4:02:06 PM10/6/23

to DSpace Technical Support

I probably should have mentioned that I run

sudo /dspace/bin/dspace index-discovery -b

on the new server first and it went OK.

After that I did

sudo /dspace/bin/dspace filter-media
and then

sudo /dspace/bin/dspace index-discovery

throws the error java.lang.IllegalArgumentException: More than one result found

DSpace Technical Support

unread,

Oct 16, 2023, 4:13:11 PM10/16/23

to DSpace Technical Support

Hi Andrew,

Apologies for not getting back to this. I'm not always able to answer questions on this list quickly, but I see no one else has gotten to this one.

The "More than one result found" error appears to be saying that somehow you have *two* WorkspaceItem objects that link to the same Item. In other words, somehow you have two entries in the "workspaceitem" table which have the *same value* in the "item_id" column. That's not supposed to be possible in DSpace, as every workspace Item should be linked to a different Item.

You should be able to find the duplicate entry by running something like this:

select item_id, COUNT(item_id) from workspaceitem group by item_id having count(item_id)>1;

That should return duplicate "item_id" values. Then, for each, you'd have to find the duplicate rows:

select * from workspaceitem where item_id=[value];

It's unclear to me why you have duplicates, but there should not be two rows with the same "item_id". So, you should delete one of them.

It's difficult to answer your other question about the "psql" role. It might be possible to ignore if that role is no longer necessary in your new setup. But, if you notice odd permissions errors, then it's possible that role is needed.

Tim

Andrew K

unread,

Oct 19, 2023, 1:39:53 PM10/19/23

to DSpace Technical Support

Hello Tim,

Thank you! It did the job perfectly. I found and deleted 3 duplicate items.

Another question now.

All the imported items have URI http://oldserver/jspui/handle/123456789/12345

But I need them https://newserver/handle/123456789/12345

What is the best way to replace all at once?

Thanks,

Andrew

понеділок, 16 жовтня 2023 р. о 23:13:11 UTC+3 DSpace Technical Support пише:

DSpace Technical Support

unread,

Oct 19, 2023, 3:50:44 PM10/19/23

to DSpace Technical Support

Hi Andrew,

I'm assuming you are looking at the URI from the DSpace 7 User Interface? If so, that's the "dc.identifier.uri" on that particular object. So you'd want to update your "dc.identifier.uri" field values. You could do this in bulk for Items by using the Batch Metadata Editing tool: https://wiki.lyrasis.org/display/DSDOC7x/Batch+Metadata+Editing

Or, there are other threads that point out how to do this at the database level (via SQL):

For example, this one on dspace-devel: https://groups.google.com/g/dspace-devel/c/y-Mj8D2xpGM

And this one on dspace-tech: https://groups.google.com/g/dspace-tech/c/njX68ISjJ3s/m/VuiUYmDvAQAJ

Definitely keep in mind that some questions come up frequently, and it's worth a quick search or two to see if you can find the answer already on these lists.

Tim

Reply all

Reply to author

Forward