Dspace 7.4 Index-discovery -b ERROR

[Juan López]

Hi!

I've just installed dspace 7.4 and everything is working fine except the index-discovery and index-authority that are showing some java.lang.NullPointerException.

My dspace is missing over 20k documents because of this... I've attached both errors on this message, hope you guys can help with this! 

Best regards,

Juan.

[Tim Donohue]

Hi Juan,

I think we're going to need more information about your setup, as this error on its own doesn't provide us with enough information to help.  Are you upgrading to 7.4 from a prior release (if so what version)?  Did you notice any other errors in your dspace.log file before​ you triggered the reindex?  

It almost sounds like you have database migrations that didn't run or didn't complete successfully.  So you also may want to verify that all your migrations were successful (check the output of "./dspace database info").  If you are upgrading, it's very important that all database migrations run before you try indexing or similar.

I'm not seeing any obvious changes between 7.3 and 7.4 that would change the indexing behaviors of DSpace, so I suspect that there may be something else going on.

Please send any other info to this list and hopefully we can help figure out what might be going on.

Tim

---

From: dspac...@googlegroups.com <dspac...@googlegroups.com> on behalf of Juan López <juanlop...@gmail.com>
Sent: Wednesday, October 12, 2022 10:22 AM
To: DSpace Technical Support <dspac...@googlegroups.com>
Subject: [dspace-tech] Dspace 7.4 Index-discovery -b ERROR

 

--
All messages to this mailing list should adhere to the Code of Conduct: https://www.lyrasis.org/about/Pages/Code-of-Conduct.aspx
---
You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dspace-tech/700ca7d9-e3e7-4e84-b2f8-53daef7ea191n%40googlegroups.com.

[Juan López]

Hi Tim!

I'm upgrading from a 7.3 version, whenever I get this error for the index-authority right before the one that I shared:

2022-10-12 10:46:45,398 INFO  unknown unknown org.dspace.discovery.SolrServiceImpl @ anonymous::indexed_object:Item-1499b5de-2812-4af7-a896-60f64a02581c
2022-10-12 10:46:45,413 WARN  unknown unknown org.dspace.discovery.SolrServiceMetadataBrowseIndexingPlugin @ Failed to get preferred label for dc.creator
java.lang.NullPointerException: null
    at org.dspace.content.authority.ChoiceAuthorityServiceImpl.getAuthorityByFieldKeyCollection(ChoiceAuthorityServiceImpl.java:488) ~[dspace-api-7.4.jar:7.4]
    at org.dspace.content.authority.ChoiceAuthorityServiceImpl.getLabel(ChoiceAuthorityServiceImpl.java:178) ~[dspace-api-7.4.jar:7.4]
    at org.dspace.content.authority.ChoiceAuthorityServiceImpl.getLabel(ChoiceAuthorityServiceImpl.java:173) ~[dspace-api-7.4.jar:7.4]
    at org.dspace.discovery.SolrServiceMetadataBrowseIndexingPlugin.additionalIndex(SolrServiceMetadataBrowseIndexingPlugin.java:179) [dspace-api-7.4.jar:7.4]
    at org.dspace.discovery.indexobject.IndexFactoryImpl.buildDocument(IndexFactoryImpl.java:67) [dspace-api-7.4.jar:7.4]
    at org.dspace.discovery.indexobject.DSpaceObjectIndexFactoryImpl.buildDocument(DSpaceObjectIndexFactoryImpl.java:30) [dspace-api-7.4.jar:7.4]
    at org.dspace.discovery.indexobject.ItemIndexFactoryImpl.buildDocument(ItemIndexFactoryImpl.java:139) [dspace-api-7.4.jar:7.4]
    at org.dspace.discovery.indexobject.ItemIndexFactoryImpl.buildDocument(ItemIndexFactoryImpl.java:76) [dspace-api-7.4.jar:7.4]
    at org.dspace.discovery.SolrServiceImpl.update(SolrServiceImpl.java:165) [dspace-api-7.4.jar:7.4]
    at org.dspace.discovery.SolrServiceImpl.indexContent(SolrServiceImpl.java:155) [dspace-api-7.4.jar:7.4]
    at org.dspace.discovery.SolrServiceImpl.updateIndex(SolrServiceImpl.java:340) [dspace-api-7.4.jar:7.4]
    at org.dspace.discovery.SolrServiceImpl.updateIndex(SolrServiceImpl.java:327) [dspace-api-7.4.jar:7.4]
    at org.dspace.discovery.SolrServiceImpl.createIndex(SolrServiceImpl.java:295) [dspace-api-7.4.jar:7.4]
    at org.dspace.discovery.IndexClient.internalRun(IndexClient.java:72) [dspace-api-7.4.jar:7.4]
    at org.dspace.scripts.DSpaceRunnable.run(DSpaceRunnable.java:104) [dspace-api-7.4.jar:7.4]
    at org.dspace.app.launcher.ScriptLauncher.executeScript(ScriptLauncher.java:149) [dspace-api-7.4.jar:7.4]
    at org.dspace.app.launcher.ScriptLauncher.handleScript(ScriptLauncher.java:131) [dspace-api-7.4.jar:7.4]
    at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:98) [dspace-api-7.4.jar:7.4]
2022-10-12 10:46:45,415 WARN  unknown unknown org.dspace.discovery.SolrServiceMetadataBrowseIndexingPlugin @ Failed to get variants for dc_creator
java.lang.NullPointerException: null
    at org.dspace.content.authority.ChoiceAuthorityServiceImpl.getAuthorityByFieldKeyCollection(ChoiceAuthorityServiceImpl.java:488) ~[dspace-api-7.4.jar:7.4]
    at org.dspace.content.authority.ChoiceAuthorityServiceImpl.getVariants(ChoiceAuthorityServiceImpl.java:205) ~[dspace-api-7.4.jar:7.4]
    at org.dspace.discovery.SolrServiceMetadataBrowseIndexingPlugin.additionalIndex(SolrServiceMetadataBrowseIndexingPlugin.java:203) [dspace-api-7.4.jar:7.4]
    at org.dspace.discovery.indexobject.IndexFactoryImpl.buildDocument(IndexFactoryImpl.java:67) [dspace-api-7.4.jar:7.4]
    at org.dspace.discovery.indexobject.DSpaceObjectIndexFactoryImpl.buildDocument(DSpaceObjectIndexFactoryImpl.java:30) [dspace-api-7.4.jar:7.4]
    at org.dspace.discovery.indexobject.ItemIndexFactoryImpl.buildDocument(ItemIndexFactoryImpl.java:139) [dspace-api-7.4.jar:7.4]
    at org.dspace.discovery.indexobject.ItemIndexFactoryImpl.buildDocument(ItemIndexFactoryImpl.java:76) [dspace-api-7.4.jar:7.4]
    at org.dspace.discovery.SolrServiceImpl.update(SolrServiceImpl.java:165) [dspace-api-7.4.jar:7.4]
    at org.dspace.discovery.SolrServiceImpl.indexContent(SolrServiceImpl.java:155) [dspace-api-7.4.jar:7.4]
    at org.dspace.discovery.SolrServiceImpl.updateIndex(SolrServiceImpl.java:340) [dspace-api-7.4.jar:7.4]
    at org.dspace.discovery.SolrServiceImpl.updateIndex(SolrServiceImpl.java:327) [dspace-api-7.4.jar:7.4]
    at org.dspace.discovery.SolrServiceImpl.createIndex(SolrServiceImpl.java:295) [dspace-api-7.4.jar:7.4]
    at org.dspace.discovery.IndexClient.internalRun(IndexClient.java:72) [dspace-api-7.4.jar:7.4]
    at org.dspace.scripts.DSpaceRunnable.run(DSpaceRunnable.java:104) [dspace-api-7.4.jar:7.4]
    at org.dspace.app.launcher.ScriptLauncher.executeScript(ScriptLauncher.java:149) [dspace-api-7.4.jar:7.4]
    at org.dspace.app.launcher.ScriptLauncher.handleScript(ScriptLauncher.java:131) [dspace-api-7.4.jar:7.4]
    at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:98) [dspace-api-7.4.jar:7.4]
2022-10-12 10:46:45,474 ERROR unknown unknown org.dspace.scripts.handler.impl.CommandLineDSpaceRunnableHandler @ Null value was assigned to a property [class org.dspace.content.MetadataValue.place] of primitive type setter of org.dspace.content.MetadataValue.place
org.hibernate.PropertyAccessException: Null value was assigned to a property [class org.dspace.content.MetadataValue.place] of primitive type setter of org.dspace.content.MetadataValue.place
    at org.hibernate.property.access.spi.SetterFieldImpl.set(SetterFieldImpl.java:58) ~[hibernate-core-5.6.5.Final.jar:5.6.5.Final]

This is my ./dspace database-info output

+-----------+----------------+------------------------------------------------------+----------+---------------------+----------------+
| Category  | Version        | Description                                          | Type     | Installed On        | State          |
+-----------+----------------+------------------------------------------------------+----------+---------------------+----------------+
| Versioned | 1.1            | Initial DSpace 1.1 database schema                   | SQL      |                     | Below Baseline |
| Versioned | 1.2            | Upgrade to DSpace 1.2 schema                         | SQL      |                     | Below Baseline |
| Versioned | 1.3            | Upgrade to DSpace 1.3 schema                         | SQL      |                     | Below Baseline |
| Versioned | 1.3.9          | Drop constraint for DSpace 1 4 schema                | JDBC     |                     | Below Baseline |
| Versioned | 1.4            | Upgrade to DSpace 1.4 schema                         | SQL      |                     | Below Baseline |
| Versioned | 1.5            | Upgrade to DSpace 1.5 schema                         | SQL      |                     | Below Baseline |
| Versioned | 1.5.9          | Drop constraint for DSpace 1 6 schema                | JDBC     |                     | Below Baseline |
| Versioned | 1.6            | Upgrade to DSpace 1.6 schema                         | SQL      |                     | Below Baseline |
| Versioned | 1.7            | Upgrade to DSpace 1.7 schema                         | SQL      |                     | Below Baseline |
| Versioned | 1.8            | Upgrade to DSpace 1.8 schema                         | SQL      |                     | Below Baseline |
| Versioned | 3.0            | Upgrade to DSpace 3.x schema                         | SQL      |                     | Below Baseline |
|           | 4.0            | Initializing from DSpace 4.0 database schema         | BASELINE | 2017-03-23 17:45:08 | Baseline       |
| Versioned | 4.9.2015.10.26 | DS-2818 registry update                              | SQL      | 2017-03-23 17:45:08 | Success        |
| Versioned | 5.0.2014.08.08 | DS-1945 Helpdesk Request a Copy                      | SQL      | 2017-03-23 17:45:08 | Success        |
| Versioned | 5.0.2014.09.25 | DS 1582 Metadata For All Objects drop constraint     | JDBC     | 2017-03-23 17:45:08 | Success        |
| Versioned | 5.0.2014.09.26 | DS-1582 Metadata For All Objects                     | SQL      | 2017-03-23 17:45:08 | Success        |
| Versioned | 5.6.2016.08.23 | DS-3097                                              | SQL      | 2017-03-23 17:45:36 | Success        |
| Versioned | 6.0.2015.03.06 | DS 2701 Dso Uuid Migration                           | JDBC     | 2017-03-23 17:45:36 | Success        |
| Versioned | 6.0.2015.03.07 | DS-2701 Hibernate migration                          | SQL      | 2017-03-23 17:45:36 | Success        |
| Versioned | 6.0.2015.08.31 | DS 2701 Hibernate Workflow Migration                 | JDBC     | 2017-03-23 17:46:58 | Success        |
| Versioned | 6.0.2016.01.03 | DS-3024                                              | SQL      | 2017-03-23 17:46:58 | Success        |
| Versioned | 6.0.2016.01.26 | DS 2188 Remove DBMS Browse Tables                    | JDBC     | 2017-03-23 17:46:59 | Success        |
| Versioned | 6.0.2016.02.25 | DS-3004-slow-searching-as-admin                      | SQL      | 2017-03-23 17:46:59 | Success        |
| Versioned | 6.0.2016.04.01 | DS-1955 Increase embargo reason                      | SQL      | 2017-03-23 17:46:59 | Success        |
| Versioned | 6.0.2016.04.04 | DS-3086-OAI-Performance-fix                          | SQL      | 2017-03-23 17:46:59 | Success        |
| Versioned | 6.0.2016.04.14 | DS-3125-fix-bundle-bitstream-delete-rights           | SQL      | 2017-03-23 17:47:02 | Success        |
| Versioned | 6.0.2016.05.10 | DS-3168-fix-requestitem item id column               | SQL      | 2017-03-23 17:47:04 | Success        |
| Versioned | 6.0.2016.07.21 | DS-2775                                              | SQL      | 2017-03-23 17:47:07 | Success        |
| Versioned | 6.0.2016.07.26 | DS-3277 fix handle assignment                        | SQL      | 2017-03-23 17:47:07 | Success        |
| Versioned | 6.0.2016.08.23 | DS-3097                                              | SQL      | 2017-03-23 17:47:07 | Success        |
| Versioned | 6.1.2017.01.03 | DS 3431 Add Policies for BasicWorkflow               | JDBC     | 2019-09-11 19:10:25 | Success        |
| Versioned | 5.7.2017.04.11 | DS-3563 Index metadatavalue resource type id column  | SQL      | 2022-08-04 17:45:08 | Out of Order   |
| Versioned | 5.0.2014.11.04 | Enable XMLWorkflow Migration                         | JDBC     | 2022-08-04 12:21:42 | Out of Order   |
| Versioned | 5.7.2017.05.05 | DS 3431 Add Policies for BasicWorkflow               | JDBC     | 2022-08-04 17:45:08 | Out of Order   |
| Versioned | 6.0.2015.09.01 | DS 2701 Enable XMLWorkflow Migration                 | JDBC     | 2022-08-04 13:05:03 | Out of Order   |
| Versioned | 7.0.2017.10.12 | DS-3542-stateless-sessions                           | SQL      | 2022-08-04 13:05:05 | Success        |
| Versioned | 7.0.2018.04.03 | Upgrade Workflow Policy                              | JDBC     | 2022-08-04 13:05:05 | Success        |
| Versioned | 7.0.2018.04.16 | dspace-entities                                      | SQL      | 2022-08-04 13:05:06 | Success        |
| Versioned | 7.0.2018.06.07 | DS-3851-permission                                   | SQL      | 2022-08-04 13:05:06 | Success        |
| Versioned | 7.0.2019.05.02 | DS-4239-workflow-xml-migration                       | SQL      | 2022-08-04 13:05:06 | Success        |
| Versioned | 7.0.2019.06.14 | scripts-and-process                                  | SQL      | 2022-08-04 13:05:06 | Success        |
| Versioned | 7.0.2019.07.31 | Retrieval of name variant                            | SQL      | 2022-08-04 13:05:06 | Success        |
| Versioned | 7.0.2019.11.13 | relationship type copy left right                    | SQL      | 2022-08-04 13:05:06 | Success        |
| Versioned | 7.0.2020.01.08 | DS-626-statistics-tracker                            | SQL      | 2022-08-04 13:05:06 | Success        |
| Versioned | 7.0.2020.10.31 | CollectionCommunity Metadata Handle                  | JDBC     | 2022-08-04 13:05:06 | Success        |
| Versioned | 7.0.2021.01.22 | Remove basic workflow                                | SQL      | 2022-08-04 13:05:06 | Success        |
| Versioned | 7.0.2021.02.08 | tilted rels                                          | SQL      | 2022-08-04 13:05:06 | Success        |
| Versioned | 7.0.2021.03.18 | Move entity type to dspace schema                    | SQL      | 2022-08-04 13:05:06 | Success        |
| Versioned | 7.0.2021.09.24 | Move entity type from item template to collection    | SQL      | 2022-08-04 13:05:06 | Success        |
| Versioned | 7.0.2021.10.04 | alter collection table drop workflow stem columns    | SQL      | 2022-08-04 13:05:06 | Success        |
| Versioned | 7.1.2021.10.18 | Fix MDV place after migrating from DSpace 5          | SQL      | 2022-08-04 13:05:06 | Success        |
| Versioned | 7.3.2022.04.29 | orcid queue and history                              | SQL      | 2022-08-04 13:05:32 | Success        |
| Versioned | 7.3.2022.05.16 | Orcid token table                                    | SQL      | 2022-08-04 13:05:32 | Success        |
| Versioned | 7.3.2022.06.16 | process to group                                     | SQL      | 2022-08-04 13:05:32 | Success        |
| Versioned | 7.3.2022.06.20 | add last version status column to relationship table | SQL      | 2022-08-04 13:05:32 | Success        |

Sorry for the long reply!

[Tim Donohue]

Hi Juan,

Regarding the error thrown by "index-discovery -b", it almost sounds like you have entries in your "metadatavalue" database table which have a NULL value for the "place" column. 

That doesn't make sense because that column should be an integer.

I guess you could see if this query against your database returns anything:

select * from metadatavalue where place is null;

The "place" column should be an integer (starting with zero) which specifies the ordering of values in a metadatafield.  This behavior also existed in 7.3 though, so it's confusing why 7.3 would not​ also throw this same error for you.  I don't see any changes to the codebase between 7.3 and 7.4 that would cause a different indexing behavior (though it's possible I'm overlooking something).

Just to double check, I've verified locally that the "./dspace index-discovery -b" command works fine for me.  I've also verified that I get no results when I run "select * from metadatavalue where place is null;" on my database.

I have NOT tested the index-authority script at all yet...but, I suspect the bug you are getting there is caused by the issue with "index-discovery -b".

Tim

---

From: dspac...@googlegroups.com <dspac...@googlegroups.com> on behalf of Juan López <juanlop...@gmail.com>

Sent: Wednesday, October 12, 2022 10:53 AM

To: DSpace Technical Support <dspac...@googlegroups.com>

Subject: Re: [dspace-tech] Dspace 7.4 Index-discovery -b ERROR

 

To view this discussion on the web visit https://groups.google.com/d/msgid/dspace-tech/f39aba43-87ed-4eda-aac8-d0845eeb16can%40googlegroups.com.

[Juan López]

Hi Tim,

You're right! we had some place values with null instead of an integer!

"Index-discovery -b" It's working right now, I'll try to run an index-authority and see if that's working too.

[Juan López]

Ok, index-authority is not working, it keeps showing the same error.

This is what my solrauthority.cfg looks like:

#---------------------------------------------------------------#

#---------------SOLR AUTHORITY CONFIGURATIONS-------------------#

#---------------------------------------------------------------#

# These configs are only used by the SOLR authority index       #

#---------------------------------------------------------------#

# Solr Authority index location

# Default is ${solr.server}/authority, unless solr.multicorePrefix is specified

; solr.authority.server=${solr.server}/authority

solr.authority.server=${solr.server}/${solr.multicorePrefix}authority

choices.plugin.dc.creator = SolrAuthorAuthority

choices.presentation.dc.creator = authorLookup

authority.controlled.dc.creator = true

authority.author.indexer.field.1=dc.creator

authority.minconfidence = uncertain

authority.minconfidence.dc.creator = uncertain

choices.plugin.dc.subject.ddc = ddc_es

choices.presentation.dc.subject.ddc = lookup

choices.closed.dc.subject.ddc = true

authority.controlled.dc.subject.ddc = true

authority.required.dc.subject.ddc = true

vocabulary.plugin.ddc_es.hierarchy.store = false

#vocabulary.plugin.ddc_es.hierarchy.suggest = true

#vocabulary.plugin.ddc_es.delimiter = "::"

# --- dc.contributor.advisor --- #

choices.plugin.dc.contributor.advisor = SolrAuthorAuthority

choices.presentation.dc.contributor.advisor = authorLookup

authority.controlled.dc.contributor.advisor = true

authority.author.indexer.field.2 = dc.contributor.advisor

authority.minconfidence = uncertain

authority.minconfidence.dc.contributor.advisor = uncertain

# --- dc.contributor.advisor --- #

choices.plugin.dc.contributor.advisor = SolrAuthorAuthority

choices.presentation.dc.contributor.advisor = authorLookup

authority.controlled.dc.contributor.advisor = true

authority.author.indexer.field.4 = dc.contributor.advisor

authority.minconfidence = uncertain

authority.minconfidence.dc.contributor.advisor = uncertain

# --- dc.contributor.author --- #

choices.plugin.dc.contributor.author = SolrAuthorAuthority

choices.presentation.dc.contributor.author = authorLookup

authority.controlled.dc.contributor.author = true

authority.author.indexer.field.5 = dc.contributor.author

authority.minconfidence.dc.contributor.author = uncertain

# --- dc.contributor.coadvisor --- #

choices.plugin.dc.contributor.coadvisor = SolrAuthorAuthority

choices.presentation.dc.contributor.coadvisor = authorLookup

authority.controlled.dc.contributor.coadvisor = true

authority.author.indexer.field.6 = dc.contributor.coadvisor

authority.minconfidence.dc.contributor.coadvisor = uncertain

# --- dc.contributor.editor --- #

choices.plugin.dc.contributor.editor = SolrAuthorAuthority

choices.presentation.dc.contributor.editor = authorLookup

authority.controlled.dc.contributor.editor = true

authority.author.indexer.field.7 = dc.contributor.editor

authority.minconfidence.dc.contributor.editor = uncertain

# --- dc.contributor.illustrator --- #

choices.plugin.dc.contributor.illustrator = SolrAuthorAuthority

choices.presentation.dc.contributor.illustrator = authorLookup

authority.controlled.dc.contributor.illustrator = true

authority.author.indexer.field.8 = dc.contributor.illustrator

authority.minconfidence.dc.contributor.illustrator = uncertain

# --- dc.contributor.other --- #

choices.plugin.dc.contributor.other = SolrAuthorAuthority

choices.presentation.dc.contributor.other = authorLookup

authority.controlled.dc.contributor.other = true

authority.author.indexer.field.9 = dc.contributor.other

authority.minconfidence.dc.contributor.other = uncertain

# --- dc.contributor.presenter --- #

choices.plugin.dc.contributor.presenter = SolrAuthorAuthority

choices.presentation.dc.contributor.presenter = authorLookup

authority.controlled.dc.contributor.presenter = true

authority.author.indexer.field.10 = dc.contributor.presenter

authority.minconfidence.dc.contributor.presenter = uncertain

# --- dc.contributor.tutor --- #

choices.plugin.dc.contributor.tutor = SolrAuthorAuthority

choices.presentation.dc.contributor.tutor = authorLookup

authority.controlled.dc.contributor.tutor = true

authority.author.indexer.field.11 = dc.contributor.tutor

authority.minconfidence.dc.contributor.tutor = uncertain

# --- dc.contributor --- #

choices.plugin.dc.contributor = SolrAuthorAuthority

choices.presentation.dc.contributor = authorLookup

authority.controlled.dc.contributor = true

authority.author.indexer.field.12 = dc.contributor

authority.minconfidence.dc.contributor = uncertain

Does this file has something wrong?

[Juan López]

I got it working, I was using  EPersonauthority instead of SolrAuthority.