[DuraSpace JIRA] (DS-3658) Full reindex (index-discovery) is triggered whenever Flyway finds a new DB migration

[Tim Donohue (DuraSpace JIRA)]

Tim Donohue updated an issue

DSpace / DS-3658 Full reindex (index-discovery) is triggered whenever Flyway finds a new DB migration Change By: Tim Donohue Summary: discovery- Full reindex ( index runs anytime -discovery) is triggered whenever Flyway finds a new DB migration , make this process more intelligent Add Comment

This message was sent by Atlassian JIRA (v7.3.3#73014-sha1:d5be8da)

[Tim Donohue (DuraSpace JIRA)]

Tim Donohue commented on DS-3658

Re: Full reindex (index-discovery) is triggered whenever Flyway finds a new DB migration How this occurs is as follows: After a migration, DatabaseUtils.setReindexDiscovery(true) is called: https://github.com/DSpace/DSpace/blob/master/dspace-api/src/main/java/org/dspace/storage/rdbms/DatabaseUtils.java#L658  This creates an empty file "[dspace]/solr/search/conf/reindex.flag" : https://github.com/DSpace/DSpace/blob/master/dspace-api/src/main/java/org/dspace/storage/rdbms/DatabaseUtils.java#L1206  The next time you start up Tomcat, the DatabaseUtils.checkIndexDiscovery() method is triggered. It checks for this "reindex.flag" file. https://github.com/DSpace/DSpace/blob/master/dspace-api/src/main/java/org/dspace/storage/rdbms/DatabaseUtils.java#L1265 If that "reindex.flag" file exists, then a new ReindexThread is created/started, to do a full reindex (afterwards that "reindex.flag" file is deleted) https://github.com/DSpace/DSpace/blob/master/dspace-api/src/main/java/org/dspace/storage/rdbms/DatabaseUtils.java#L1281 However, if that "reindex.flag" file doesn't exist, nothing happens.

Add Comment

[Sven Soliman (DuraSpace JIRA)]

Sven Soliman commented on DS-3658

Re: Full reindex (index-discovery) is triggered whenever Flyway finds a new DB migration

I needed to Full-Index as well when upgrading from 6.0 to 6.1 and it took 5 hours ..

Add Comment

[Tim Donohue (DuraSpace JIRA)]

Tim Donohue updated an issue

DSpace / DS-3658

Full reindex (index-discovery) is triggered whenever Flyway finds a new DB migration

Change By: Tim Donohue Status: Received Volunteer Needed

Add Comment

[Mark Diggory (DuraSpace JIRA)]

Mark Diggory commented on DS-3658

Re: Full reindex (index-discovery) is triggered whenever Flyway finds a new DB migration I feel very strongly that this code should be removed (or made an option that is disabled by DEFAULT). It should be left up to the individual installing the update of the repository to decide if further steps that will be taken. Large production Solr search indexes take a long time (entire days) to reconstruct on many of our repositories, they should not be completely destroyed and rebuilt from scratch without human input.

Add Comment

This message was sent by Atlassian JIRA (v7.10.0#710001-sha1:0399717)

[Mark Diggory (DuraSpace JIRA)]

Mark Diggory edited a comment on DS-3658

Re: Full reindex (index-discovery) is triggered whenever Flyway finds a new DB migration

I feel very strongly that [this code|https://github.com/DSpace/DSpace/blame/master/dspace-api/src/main/java/org/dspace/storage/rdbms/DatabaseUtils.java#L1216-L1256] should be removed (or made an option that is disabled by DEFAULT). It should be left up to the individual installing the update of the repository to decide if further steps that will be taken. Large production Solr search indexes take a long time (entire days) to reconstruct on many of our repositories, they should not be completely destroyed and rebuilt from scratch without human input. I would raise the priority of this and identify it as an important issue that should be corrected.

Add Comment

[Andrew Wood Atmire (DuraSpace JIRA)]

Andrew Wood Atmire commented on DS-3658

Re: Full reindex (index-discovery) is triggered whenever Flyway finds a new DB migration

I've created a PR that wraps the automatic invocation of the ReindexerThread with a configurable property. Effectively allowing the user to configure the automatic triggering of the ReindexerThread   https://github.com/DSpace/DSpace/pull/2184

Add Comment

[Anonymous (DuraSpace JIRA)]

Issue was automatically transitioned when Andrew Wood created pull request #2184 in GitHub

DSpace / DS-3658

Full reindex (index-discovery) is triggered whenever Flyway finds a new DB migration

Change By: Andrew Wood Status: Volunteer Code Review Needed

Add Comment

[Mark H. Wood (DuraSpace JIRA)]

Mark H. Wood updated an issue

DSpace / DS-3658 Full reindex (index-discovery) is triggered whenever Flyway finds a new DB migration

Change By: Mark H. Wood Fix Version/s: 7.0 Fix Version/s: 6.4

Add Comment

[Mark H. Wood (DuraSpace JIRA)]

Mark H. Wood commented on DS-3658

Re: Full reindex (index-discovery) is triggered whenever Flyway finds a new DB migration Draft 7.0 documentation updated.  6_x documentation should not be updated until 6.4 is released, and should note that this feature did not exist before that release.

Add Comment

[Tim Donohue (DuraSpace JIRA)]

Tim Donohue updated an issue

DSpace / DS-3658

Full reindex (index-discovery) is triggered whenever Flyway finds a new DB migration

Change By: Tim Donohue Status: Code Review Needed Awaiting Documentation

Add Comment

[Anonymous (DuraSpace JIRA)]

Issue was automatically transitioned when pull request #2433 was created in GitHub

DSpace / DS-3658 Full reindex (index-discovery) is triggered whenever Flyway finds a new DB migration

Change By: Automated Transition Status: Awaiting Documentation Code Review Needed

Add Comment

[Tim Donohue (DuraSpace JIRA)]

Tim Donohue updated an issue

DSpace / DS-3658 Full reindex (index-discovery) is triggered whenever Flyway finds a new DB migration

Change By: Tim Donohue

Status: Code Review Needed Awaiting Documentation

Add Comment

[Anonymous (DuraSpace JIRA)]

Issue was automatically transitioned when pull request #2441 was created in GitHub

DSpace / DS-3658 Full reindex (index-discovery) is triggered whenever Flyway finds a new DB migration

Change By: Automated Transition

Status: Awaiting Documentation Code Review Needed

Add Comment

[Kim Shepherd (DuraSpace JIRA)]

Kim Shepherd updated an issue

DSpace / DS-3658 Full reindex (index-discovery) is triggered whenever Flyway finds a new DB migration

switching status to remind us about the need for 6.x doc before 6.4 is released Change By: Kim Shepherd

Status: Code Review Needed Awaiting Documentation

Add Comment

This message was sent by Atlassian Jira (v8.4.1#804002-sha1:94e96d6)

[Tim Donohue (LYRASIS JIRA)]

Tim Donohue closed an issue as Fixed

In order to close out the "7.0" Version in Jira (so that nothing new can be added to it), I need to close this ticket as fixed in 7.0. I realize it's (possibly) still being documented in 6.4, but that will need to occur in a separate Jira ticket as needed.  Sorry about that.

DSpace / DS-3658 Full reindex (index-discovery) is triggered whenever Flyway finds a new DB migration

Change By: Tim Donohue Resolution: Fixed Status: Awaiting Documentation Closed

Add Comment

This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)