potential bug in harvester

[Jacek Chudzik]

Hi,

I've noticed something on Dataverse v.6.9.

It seems that there is a sync problem with harvesting, that regards datasets with changed/deleted files.

In our case we have:

- created and published dataset with 2 files

- dataset was harvested properly

- 1 file was deleted and other 3 files were added and dataset was published as version 2.0

- harvesting fails

In harvester log I see:

edu.harvard.iq.dataverse.api.imports.ImportException, Failed to import harvested dataset: class edu.harvard.iq.dataverse.engine.command.exception.CommandException (Command edu.harvard.iq.dataverse.engine.command.impl.UpdateHarvestedDatasetCommand@3c3a28ff failed: Exception thrown from bean: java.lang.IllegalArgumentException: Cannot merge an Entity that has been removed: edu.harvard.iq.dataverse.FileMetadata[id=92619])

So it seems that file removal might be the real problem here. 

I have removed harvesting client for this collection and then I've made a new one (to remove dataset from harvester). After this step harvesting works fine as it has no memory of removed files.

Is this a bug?

We have just noticed that but in time there might be a problem with growing difference in number of files between Dataverse instance and harvesting instance.

Regards,

Jacek

[Philip Durbin]

Hi Jacek,

Yes, it's a bug and we are already tracking it at https://github.com/IQSS/dataverse/issues/11933

Please feel free to comment on it with your detailed steps on how to reproduce it!

Thanks,

Phil

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/dataverse-community/5c22027f-cc4b-4ed8-9a87-8f79af236332n%40googlegroups.com.

--

Philip Durbin
Software Developer for http://dataverse.org
http://www.iq.harvard.edu/people/philip-durbin