Database upgrade error / Dspace 7 loading continuously

[Lewatle Johannes Phaladi]

Dear DSpace Team,

Please assist I am now getting the following error, when running [dspace]/bin/dspace database migrate, see terminal results below also see attachment.

-bash-4.2$ ./dspace database migrate

Database URL: jdbc:postgresql://localhost:5432/wiredspacexra
Migrating database to latest version... (Check dspace logs for details)
Migration exception:
java.sql.SQLException: Flyway migration error occurred
    at org.dspace.storage.rdbms.DatabaseUtils.updateDatabase(DatabaseUtils.java:738)
    at org.dspace.storage.rdbms.DatabaseUtils.updateDatabase(DatabaseUtils.java:632)
    at org.dspace.storage.rdbms.DatabaseUtils.main(DatabaseUtils.java:228)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.dspace.app.launcher.ScriptLauncher.runOneCommand(ScriptLauncher.java:277)
    at org.dspace.app.launcher.ScriptLauncher.handleScript(ScriptLauncher.java:133)
    at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:98)
Caused by: org.flywaydb.core.internal.command.DbMigrate$FlywayMigrateException:
Migration V7.0_2019.05.02__DS-4239-workflow-xml-migration.sql failed
--------------------------------------------------------------------
SQL State  : 42P01
Error Code : 0
Message    : ERROR: relation "cwf_pooltask" does not exist
  Position: 8
Location   : org/dspace/storage/rdbms/sqlmigration/postgres/V7.0_2019.05.02__DS-4239-workflow-xml-migration.sql (/dspace/Install/dspace-backend/bin/file:/dspace/Install/dspace-backend/lib/dspace-api-7.0.jar!/org/dspace/storage/rdbms/sqlmigration/postgres/V7.0_2019.05.02__DS-4239-workflow-xml-migration.sql)
Line       : 16
Statement  : UPDATE cwf_pooltask SET workflow_id='defaultWorkflow' WHERE workflow_id='default'

    at org.flywaydb.core.internal.command.DbMigrate.doMigrateGroup(DbMigrate.java:369)
    at org.flywaydb.core.internal.command.DbMigrate.access$200(DbMigrate.java:54)
    at org.flywaydb.core.internal.command.DbMigrate$3.call(DbMigrate.java:282)
    at org.flywaydb.core.internal.jdbc.TransactionalExecutionTemplate.execute(TransactionalExecutionTemplate.java:66)
    at org.flywaydb.core.internal.command.DbMigrate.applyMigrations(DbMigrate.java:279)
    at org.flywaydb.core.internal.command.DbMigrate.migrateGroup(DbMigrate.java:244)
    at org.flywaydb.core.internal.command.DbMigrate.access$100(DbMigrate.java:54)
    at org.flywaydb.core.internal.command.DbMigrate$2.call(DbMigrate.java:162)
    at org.flywaydb.core.internal.command.DbMigrate$2.call(DbMigrate.java:159)
    at org.flywaydb.core.internal.database.postgresql.PostgreSQLAdvisoryLockTemplate.execute(PostgreSQLAdvisoryLockTemplate.java:71)
    at org.flywaydb.core.internal.database.postgresql.PostgreSQLConnection.lock(PostgreSQLConnection.java:99)
    at org.flywaydb.core.internal.schemahistory.JdbcTableSchemaHistory.lock(JdbcTableSchemaHistory.java:140)
    at org.flywaydb.core.internal.command.DbMigrate.migrateAll(DbMigrate.java:159)
    at org.flywaydb.core.internal.command.DbMigrate.migrate(DbMigrate.java:137)
    at org.flywaydb.core.Flyway$1.execute(Flyway.java:206)
    at org.flywaydb.core.Flyway$1.execute(Flyway.java:158)
    at org.flywaydb.core.Flyway.execute(Flyway.java:527)
    at org.flywaydb.core.Flyway.migrate(Flyway.java:158)
    at org.dspace.storage.rdbms.DatabaseUtils.updateDatabase(DatabaseUtils.java:729)
    ... 9 more
Caused by: org.flywaydb.core.internal.sqlscript.FlywaySqlScriptException:
Migration V7.0_2019.05.02__DS-4239-workflow-xml-migration.sql failed
--------------------------------------------------------------------
SQL State  : 42P01
Error Code : 0
Message    : ERROR: relation "cwf_pooltask" does not exist
  Position: 8
Location   : org/dspace/storage/rdbms/sqlmigration/postgres/V7.0_2019.05.02__DS-4239-workflow-xml-migration.sql (/dspace/Install/dspace-backend/bin/file:/dspace/Install/dspace-backend/lib/dspace-api-7.0.jar!/org/dspace/storage/rdbms/sqlmigration/postgres/V7.0_2019.05.02__DS-4239-workflow-xml-migration.sql)
Line       : 16
Statement  : UPDATE cwf_pooltask SET workflow_id='defaultWorkflow' WHERE workflow_id='default'

    at org.flywaydb.core.internal.sqlscript.DefaultSqlScriptExecutor.handleException(DefaultSqlScriptExecutor.java:275)
    at org.flywaydb.core.internal.sqlscript.DefaultSqlScriptExecutor.executeStatement(DefaultSqlScriptExecutor.java:220)
    at org.flywaydb.core.internal.sqlscript.DefaultSqlScriptExecutor.execute(DefaultSqlScriptExecutor.java:127)
    at org.flywaydb.core.internal.resolver.sql.SqlMigrationExecutor.executeOnce(SqlMigrationExecutor.java:88)
    at org.flywaydb.core.internal.resolver.sql.SqlMigrationExecutor.access$000(SqlMigrationExecutor.java:33)
    at org.flywaydb.core.internal.resolver.sql.SqlMigrationExecutor$1.call(SqlMigrationExecutor.java:77)
    at org.flywaydb.core.internal.resolver.sql.SqlMigrationExecutor$1.call(SqlMigrationExecutor.java:74)
    at org.flywaydb.core.internal.database.DefaultExecutionStrategy.execute(DefaultExecutionStrategy.java:28)
    at org.flywaydb.core.internal.resolver.sql.SqlMigrationExecutor.execute(SqlMigrationExecutor.java:74)
    at org.flywaydb.core.internal.command.DbMigrate.doMigrateGroup(DbMigrate.java:366)
    ... 27 more
Caused by: org.postgresql.util.PSQLException: ERROR: relation "cwf_pooltask" does not exist
  Position: 8
    at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2505)
    at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2241)
    at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:310)
    at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:447)
    at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:368)
    at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:309)
    at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:295)
    at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:272)
    at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:267)
    at org.apache.commons.dbcp2.DelegatingStatement.execute(DelegatingStatement.java:194)
    at org.apache.commons.dbcp2.DelegatingStatement.execute(DelegatingStatement.java:194)
    at org.flywaydb.core.internal.jdbc.JdbcTemplate.executeStatement(JdbcTemplate.java:244)
    at org.flywaydb.core.internal.sqlscript.ParsedSqlStatement.execute(ParsedSqlStatement.java:111)
    at org.flywaydb.core.internal.sqlscript.DefaultSqlScriptExecutor.executeStatement(DefaultSqlScriptExecutor.java:208)
    ... 35 more
-bash-4.2$ 

Regards,

Lewatle

[Tim Donohue]

Hi Lewatle,

Since you are upgrading​ to DSpace 7, you must run:

./dspace database migrate ignored

The "ignored" part is very important...as it ensures some previously ignored migrations (for 6.x) are run (as they are now required for 7.x).  See the upgrade step #9(c): https://wiki.lyrasis.org/display/DSDOC7x/Upgrading+DSpace

Tim

---

From: dspac...@googlegroups.com <dspac...@googlegroups.com> on behalf of Lewatle Johannes Phaladi <joem...@gmail.com>
Sent: Friday, October 1, 2021 6:04 AM
To: DSpace Technical Support <dspac...@googlegroups.com>
Subject: [dspace-tech] Database upgrade error / Dspace 7 loading continuously

 

--
All messages to this mailing list should adhere to the Code of Conduct: https://www.lyrasis.org/about/Pages/Code-of-Conduct.aspx
---
You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dspace-tech/03e667e3-6756-4c73-9e28-92ae26b71ac8n%40googlegroups.com.

[Lewatle Johannes Phaladi]

Hi Tim,

much appreciated I managed to run database migrate successfully.

But the items as well as sub-communities are not shown on user interface, I have set assetstore path in local.cfg and /dspace-backend/config/spring/api/bitstore.xml then restart tomcat 

Will that be cause by handle server as I haven't set it yet ?

Regards,

Lewatle 

[Lewatle Johannes Phaladi]

[Tim Donohue]

Hi Lewatle,

It's hard to say what exactly could be causing that problem. It might have to do with Solr not working or setup properly, or it could even be that the frontend is having issues communicating with the backend.  

The best way to debug the issue would be to locate the exact error message in either the backend (logs) or in the frontend (your browser's devtools or by turning on REST API debugging).    Please see the guide at https://wiki.lyrasis.org/display/DSPACE/Troubleshoot+an+error#Troubleshootanerror-DSpace7.x(orabove)

Once you have an error message, check to see if it's a common issue listed in the install guide: https://wiki.lyrasis.org/display/DSDOC7x/Installing+DSpace#InstallingDSpace-CommonInstallationIssues

If you need more help, send the error message to this list and we'll give more advice.

Tim

---

From: dspac...@googlegroups.com <dspac...@googlegroups.com> on behalf of Lewatle Johannes Phaladi <joem...@gmail.com>

Sent: Monday, October 4, 2021 8:28 AM

To: DSpace Technical Support <dspac...@googlegroups.com>

Subject: Re: [dspace-tech] Database upgrade error / Dspace 7 loading continuously

 

To view this discussion on the web visit https://groups.google.com/d/msgid/dspace-tech/0edbe510-b081-480c-a572-dd82087a4e70n%40googlegroups.com.

[Lewatle Johannes Phaladi]

Hi Tim,

Thanks for the reply and indeed in the backend log file it shows that there is error  on SOLR, also tomcat started to go down after some time while still trying to fix issue of items not shown in DSpace web interface, I have attached 2 catalina log files and dspace log,   

Regards,

Lewatle 

[Joel R]

Lewatle,

Did you see this error?

12-Oct-2021 12:54:42.464 SEVERE [main] org.apache.catalina.core.StandardServer.await Failed to create server shutdown socket on address [localhost] and port [8005] (base port [8005] and offset [0]) java.net.BindException: Address already in use (Bind failed)

I'm not a tomcat/java expert, but that port, 8005, should not already be in use when Tomcat starts. Is tomcat running twice? Is something else using that port? That could be a clue to what's going on. It could also mean that's why SOLR is having trouble.

Second, this is a big clue that something is wrong with SOLR:

2021-10-13 01:15:01,761 ERROR unknown unknown org.dspace.app.sitemap.GenerateSitemaps @ Failed getting entitytype through solr for item d137c57a-caff-4dde-a401-5fd658584dc0: Error from server at http://localhost:8983/solr/search: Expected mime type application/octet-stream but got text/html. <html>

What happens when you try to go to http://localhost:8983/solr/ or http://localhost:8983/ ? You should see the SOLR manager page at the first URL. Tim is right in that the DSpace front end is not able to communicate to SOLR. I also had the same sort of "Error Fetching Sub-Communities" and I resolved it in a way that was appropriate for my situation, but it was related to SOLR.

--Joel

[Lewatle Johannes Phaladi]

Hi Joel,

Thanks a lot, I look at it and come back with the reply.

Regards,

Lewatle