[Dspace-tech] problems with dspace 5.1 migration - error on eperson-firstname and unable to create administrator fresh install

Searle, Shannon

unread,

Aug 26, 2015, 2:30:22 PM8/26/15

to dspac...@lists.sourceforge.net

Hello,

I have been working on an upgrade from an older postgres db in 1.6 dspace to the latest install of dspace 5.1.

I ran the upgrades, checked migrate and started up the tomcat server. All looked well at first, but then any attempt to update profiles or show users resulted in an exception.

I ran a maven clean package, then an ant fresh install. After that I couldn’t even create an administrator:

Exception: Invalid metadata field: [eperson.lastname]

java.sql.SQLException: Invalid metadata field: [eperson.lastname]

at org.dspace.content.DSpaceObject.updateMetadata(DSpaceObject.java:122)

at org.dspace.eperson.EPerson.update(EPerson.java:1056)

at org.dspace.administer.CreateAdministrator.createAdministrator(CreateAdministrator.java:259)

at org.dspace.administer.CreateAdministrator.negotiateAdministratorDetails(CreateAdministrator.java:206)

at org.dspace.administer.CreateAdministrator.main(CreateAdministrator.java:82)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.dspace.app.launcher.ScriptLauncher.runOneCommand(ScriptLauncher.java:225)

at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:77)

Anyone know where this is coming from? I did delete my old browse indexes but I didn’t think they were needed any more…this is the same type of error I was getting on the web ui when trying to access anything to do with the eperson admin interface.

Anyone who can help – greatly appreciated. I need to get this server up and running J

Thanks!

Shannon

helix84

unread,

Aug 26, 2015, 2:30:36 PM8/26/15

to Searle, Shannon, dspac...@lists.sourceforge.net

Judging from the stacktrace, my guess is that your DB schema version doesn't correspond to your DSpace version, for some unknown reason.

There was a significant change to several DB tables in DSpace 5 where many entity attributes (including eperson last name) moved from the entity table to the metadatavalue table, where they are referenced by resource_id (e.g. eperson_id) and resource_type_id (e.g. 7 for eperson). That's how you can tell a pre-DSpace 5 database from a DSpace 5 database. Please, check that the schema corresponds to your DSpace version.

Assuming you're running DSpace 5 on a DSpace 4 database, DSpace 5 will attempt to upgrade the schema automatically to DSpace 5 (this is a new feature in DSpace 5). It will also print information about all successful and unsuccessful migrations in the log, so check that. It will also create the schema_version table which contains this information.

Regards,
~~helix84

Compulsory reading: DSpace Mailing List Etiquette
https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette

Andrea Schweer

unread,

Aug 26, 2015, 2:30:44 PM8/26/15

to Searle, Shannon, dspac...@lists.sourceforge.net

Hi,

On 12/05/15 05:19, Searle, Shannon wrote:

I have been working on an upgrade from an older postgres db in 1.6 dspace to the latest install of dspace 5.1.
I ran the upgrades, checked migrate and started up the tomcat server. All looked well at first, but then any attempt to update profiles or show users resulted in an exception.

I ran a maven clean package, then an ant fresh install. After that I couldn’t even create an administrator:

Did you really run an ant fresh_install? You should only ever do that the very first time you install DSpace. From then on, run ant update. As helix84 wrote, ant update will try to auto-upgrade your database schema; I assume fresh_install won't try to do that since it assumes the database has no tables yet.

cheers,
Andrea

-- 
Dr Andrea Schweer
IRR Technical Specialist, ITS Information Systems
The University of Waikato, Hamilton, New Zealand

Searle, Shannon

unread,

Aug 26, 2015, 2:32:02 PM8/26/15

to hel...@centrum.sk, dspac...@lists.sourceforge.net

Hi all,

New environment:

Red Hat 6.6

DSpace 5.1

Postgresql 9.3

Tomcat 7

Java 7

Ant 1.9.4

Maven 3.2

Old environment

Dspace 1.6 – no solr!

Java 6

Tomcat 5.5

Postgresql 7.4

Well, at least now I have discovered why there was no eperson.firstname/lastname value – the eperson schema was never installed into the metadataschemaregistry table. At the end of the migration, the data had moved to the appropriate tables, but the schemas for the extended Dublin core attributes and dspace.eperson.org were not there at all. I added the namespace and values into eperson metadata manually, so then it stopped generating NPE’s in Java whenever I tried to access the eperson admin interface.

My eperson pages look like the attached png now – new epeople are OK of course but can’t find the names of any of the other 900 because when the name and phone and language data was migrated into the metadatavalue table there was no metadata_field_id- so this is what happens if I try - the logs say:

2015-05-18 18:21:11,486 DEBUG org.dspace.storage.rdbms.DatabaseManager @ Running query "SELECT count(*) as epcount FROM eperson WHERE eperson_id = ? OR LOWER((select text_value from metadatavalue where resource_id=? and resource_type_id=? and metadata_field_id=?)) LIKE LOWER(?) OR LOWER((select text_value from metadatavalue where resource_id=? and resource_type_id=? and metadata_field_id=?)) LIKE LOWER(?) OR LOWER(eperson.email) LIKE LOWER(?)" with parameters: -1,-1,7,81,%%,-1,7,83,%%,%%

2015-05-18 18:21:11,489 DEBUG org.dspace.storage.rdbms.DatabaseManager @ Running query "select e.* from eperson e LEFT JOIN metadatavalue fn on (resource_id=e.eperson_id AND fn.resource_type_id=? and fn.metadata_field_id=?) LEFT JOIN metadatavalue ln on (ln.resource_id=e.eperson_id AND ln.resource_type_id=? and ln.metadata_field_id=?) WHERE e.eperson_id = ? OR LOWER(fn.text_value) LIKE LOWER(?) OR LOWER(ln.text_value) LIKE LOWER(?) OR LOWER(email) LIKE LOWER(?) ORDER BY ln.text_value, fn.text_value ASC LIMIT ? " with parameters: 7,81,7,83,-1,%%,%%,%%,15

2015-05-18 18:21:11,492 DEBUG org.dspace.storage.rdbms.DatabaseManager @ Running query "SELECT * FROM MetadataValue WHERE resource_id= ? and resource_type_id = ? ORDER BY metadata_field_id, place" with parameters: 106,7

2015-05-18 18:21:11,492 DEBUG org.dspace.storage.rdbms.DatabaseManager @ Running query "SELECT * from item where submitter_id= ? " with parameters: 106

2015-05-18 18:21:11,493 DEBUG org.dspace.storage.rdbms.DatabaseManager @ Running query "SELECT * from workflowitem where owner= ? " with parameters: 106

2015-05-18 18:21:11,493 DEBUG org.dspace.storage.rdbms.DatabaseManager @ Running query "SELECT * from tasklistitem where eperson_id= ? " with parameters: 106

2015-05-18 18:21:11,493 DEBUG org.dspace.storage.rdbms.DatabaseManager @ Running query "SELECT * from item where submitter_id= ? " with parameters: 950

2015-05-18 18:21:11,494 DEBUG org.dspace.storage.rdbms.DatabaseManager @ Running query "SELECT * from workflowitem where owner= ? " with parameters: 950

2015-05-18 18:21:11,494 DEBUG org.dspace.storage.rdbms.DatabaseManager @ Running query "SELECT * from tasklistitem where eperson_id= ? " with parameters: 950

2015-05-18 18:21:11,494 DEBUG org.dspace.storage.rdbms.DatabaseManager @ Running query "SELECT * FROM MetadataValue WHERE resource_id= ? and resource_type_id = ? ORDER BY metadata_field_id, place" with parameters: 1003,7

2015-05-18 18:21:11,494 DEBUG org.dspace.storage.rdbms.DatabaseManager @ Running query "SELECT * from item where submitter_id= ? " with parameters: 1003

2015-05-18 18:21:11,495 DEBUG org.dspace.storage.rdbms.DatabaseManager @ Running query "SELECT * from workflowitem where owner= ? " with parameters: 1003

2015-05-18 18:21:11,495 DEBUG org.dspace.storage.rdbms.DatabaseManager @ Running query "SELECT * from tasklistitem where eperson_id= ? " with parameters: 1003

2015-05-18 18:21:11,495 DEBUG org.dspace.storage.rdbms.DatabaseManager @ Running query "SELECT * FROM MetadataValue WHERE resource_id= ? and resource_type_id = ? ORDER BY metadata_field_id, place" with parameters: 251,7

2015-05-18 18:21:11,495 DEBUG org.dspace.storage.rdbms.DatabaseManager @ Running query "SELECT * from item where submitter_id= ? " with parameters: 251

2015-05-18 18:21:11,496 DEBUG org.dspace.storage.rdbms.DatabaseManager @ Running query "SELECT * from workflowitem where owner= ? " with parameters: 251

2015-05-18 18:21:11,496 DEBUG org.dspace.storage.rdbms.DatabaseManager @ Running query "SELECT * from tasklistitem where eperson_id= ? " with parameters: 251

2015-05-18 18:21:11,496 DEBUG org.dspace.storage.rdbms.DatabaseManager @ Running query "SELECT * FROM MetadataValue WHERE resource_id= ? and resource_type_id = ? ORDER BY metadata_field_id, place" with parameters: 802,7

2015-05-18 18:21:11,496 ERROR org.dspace.content.DSpaceObject @ Loading item - cannot find metadata field -1 for resourceType=7 and resourceId=802

2015-05-18 18:21:11,496 DEBUG org.dspace.storage.rdbms.DatabaseManager @ Running query "SELECT * from item where submitter_id= ? " with parameters: 802

2015-05-18 18:21:11,497 DEBUG org.dspace.storage.rdbms.DatabaseManager @ Running query "SELECT * from workflowitem where owner= ? " with parameters: 802

2015-05-18 18:21:11,497 DEBUG org.dspace.storage.rdbms.DatabaseManager @ Running query "SELECT * from tasklistitem where eperson_id= ? " with parameters: 802

2015-05-18 18:21:11,499 DEBUG org.dspace.storage.rdbms.DatabaseManager @ Running query "SELECT * FROM MetadataValue WHERE resource_id= ? and resource_type_id = ? ORDER BY metadata_field_id, place" with parameters: 681,7

2015-05-18 18:21:11,499 ERROR org.dspace.content.DSpaceObject @ Loading item - cannot find metadata field -1 for resourceType=7 and resourceId=681

2015-05-18 18:21:11,499 DEBUG org.dspace.storage.rdbms.DatabaseManager @ Running query "SELECT * from item where submitter_id= ? " with parameters: 681

2015-05-18 18:21:11,499 DEBUG org.dspace.storage.rdbms.DatabaseManager @ Running query "SELECT * from workflowitem where owner= ? " with parameters: 681

2015-05-18 18:21:11,499 DEBUG org.dspace.storage.rdbms.DatabaseManager @ Running query "SELECT * from tasklistitem where eperson_id= ? " with parameters: 681

The schema table reports that the version is updated to DSpace 5.1 but obviously something is not updated, and I have no errors in the logs that indicated that the schemas weren’t added although they quite clearly were not. The tables in my database are from an older version of postgresql 7.4 which has been imported into the latest version, so I have the issue with variable names as $1 etc for the constraints as well. I have spent literally weeks and several installations but I cannot see where the errors are happening or why the installs keep failing in one way or another.

Has anyone actually managed to update a DSpace 1.6 install to 5.1? What is the best order of the steps if so? Import database into 9.3, correct variables in constraints and run database migrate? It was suggested that running the update.sequences.sql script manually would stop some of the errors but it complains then about no webapps table existing. I have done clean installs from maven package, ant clean installs, initialising new db and importing data, or creating db and then dropping and recreating with old data and then running migrate etc – just about every scenario I could imagine.

The rest of the data seems to have imported ok (handles correct, types ok) although I have fewer bitstream registry types than expected...

Looking at the table data, I can’t see any easy way to hook up the data to the correct metadata_field_id.

This is what my schema_version table says:

1 1 1.6 Initializing from DSpace 1.6 database schema INIT Initializing from DSpace 1.6 database schema dspace 35:47.4 0 TRUE

2 2 1.7 Upgrade to DSpace 1.7 schema SQL V1.7__Upgrade_to_DSpace_1.7_schema.sql -1075634434 dspace 35:47.8 12 TRUE

3 3 1.8 Upgrade to DSpace 1.8 schema SQL V1.8__Upgrade_to_DSpace_1.8_schema.sql -93232544 dspace 35:48.6 198 TRUE

4 4 3 Upgrade to DSpace 3.x schema SQL V3.0__Upgrade_to_DSpace_3.x_schema.sql 1671414988 dspace 35:48.8 166 TRUE

5 5 4 Upgrade to DSpace 4.x schema SQL V4.0__Upgrade_to_DSpace_4.x_schema.sql 2051773448 dspace 35:49.0 45 TRUE

6 6 5.0.2014.08.08 DS-1945 Helpdesk Request a Copy SQL V5.0_2014.08.08__DS-1945__a_Copy.sql -872444192 dspace 35:49.1 8 TRUE

7 7 5.0.2014.09.25 DS 1582 Metadata For All Objects drop constraint JDBC org.dspace.storage.rdbms.migration.V5_0_201 -1 dspace 11:19.8 16 TRUE

8 8 5.0.2014.09.26 DS-1582 Metadata For All Objects SQL V5.0_2014.09.26__DS-1582_Metadatcts.sql 1043319466 dspace 11:20.7 4177 TRUE

Ready to give up now...thanks for suggestions so far.

Sadly

Shannon

e-people.png

Reply all

Reply to author

Forward