Curation tasks on DSpace 6 stoped working

[enes...@yahoo.com]

We have three DSpace 6 instances on CentOS 7.  On one instance, curation tasks are not working from either CLI, XMLUI or our custom forms. The configuration is the same on all three instances.  If we run dspace curate -t taskname -i 123456789/3770 -r - -v , we get Task: taskname not resolved.  This includes curation tasks that come out of box.

We have also tried to move setting from the curate.cfg to local.cfg, but the same issue is occurring.

Did anyone experience similar issue?

Regards,
Enes

[Tim Donohue]

Hi Enes,

I just did a basic test of Curation Tasks on our http://demo.dspace.org server.  On that server, I ran the following on the commandline:

[dspace]/bin/dspace curate -t checklinks -i 10673/5 -r - -v

And, the "checklinks" curation task ran successfully on this item: http://demo.dspace.org/xmlui/handle/10673/5 without any issues.

So, I'm wondering if maybe it's a configuration error on your end, or if only specific task(s) are failing.  But, we'd probably need you to send more information to the mailing list in order to help diagnose the issue.  In any case, I'd recommend double checking that your curation task configuration, perhaps comparing it the the out-of-the-box DSpace 6.x version:

https://github.com/DSpace/DSpace/blob/dspace-6_x/dspace/config/modules/curate.cfg

(And where the configuration exists shouldn't matter too much, but any configurations in your local.cfg *will override* the settings in your curate.cfg)

- Tim

--
You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech...@googlegroups.com.
To post to this group, send email to dspac...@googlegroups.com.
Visit this group at https://groups.google.com/group/dspace-tech.
For more options, visit https://groups.google.com/d/optout.

--

Tim Donohue
Technical Lead for DSpace & DSpaceDirect
DuraSpace.org | DSpace.org | DSpaceDirect.org

[enes...@yahoo.com]

Hi Tim,

This happens on every curation task that I run on this machine.  For example:

dspace/bin/dspace curate -t requiredmetadata -i 123456789/3770 -r - -v
Adding task: requiredmetadata
Task: requiredmetadata not resolved
Starting curation
Curating id: 123456789/3770
Ending curation. Elapsed time: 576

The same command works on the other two machines and my test machine. It was working too on this machine for couple of months and it stopped. Configurations are the same on all instances. It looks like that this problem occurs while task is getting resolved in Curator.java. 

I have even tried to rebuild, but I get the same issue.  Is there anything else that I need to check (maybe at the OS level)?

Regards,
Enes

[Tim Donohue]

Hi Enes,

To me, that error "Task: requiredmetadata not resolved", still sounds suspiciously like a configuration problem.  So, I'm wondering if your primary configuration in curate.cfg is being accidentally overridden elsewhere (e.g. in local.cfg), or perhaps if the curate.cfg files isn't being loaded into your dspace.cfg properly (e.g. via an "include" line like this: https://github.com/DSpace/DSpace/blob/dspace-6_x/dspace/config/dspace.cfg#L2005)

One thing you could try... DSpace has a commandline tool that is able to check what configurations it "sees".   Try running:

[dspace]/bin/dspace dsprop -p plugin.named.org.dspace.curate.CurationTask -r

This should output the "raw" (-r) value of the "plugin.named.org.dspace.curate.CurationTask" configuration that DSpace is loading/seeing.  In a normal, default 6.x DSpace site, this command should return an array of values like this:

[org.dspace.ctask.general.NoOpCurationTask = noop, org.dspace.ctask.general.ProfileFormats = profileformats, org.dspace.ctask.general.RequiredMetadata =

requiredmetadata, org.dspace.ctask.general.ClamScan = vscan, org.dspace.ctask.general.MicrosoftTranslator = translate, org.dspace.ctask.general.MetadataValueLinkChecker = checklinks]

This array of values is defined in the curate.cfg file.  If your site does *not* return this array of values, then something is definitely wrong in your configuration.

- Tim

[enes...@yahoo.com]

Hi Tim,

I executed:
dspace/bin/dspace dsprop -p plugin.named.org.dspace.curate.CurationTask -r

My output is:
[org.dspace.ctask.general.NoOpCurationTask = noop, org.dspace.ctask.general.ProfileFormats = profileformats, org.dspace.ctask.general.RequiredMetadata = requiredmetadata, org.dspace.ctask.general.ClamScan = vscan, org.dspace.ctask.general.MicrosoftTranslator = translate, org.dspace.ctask.general.MetadataValueLinkChecker = checklinks, org.dspace.ctask.general.ValidateMetadata = validatemetadata, org.dspace.ctask.general.ExtractMBRfromTFW = extractmbr, org.dspace.ctask.general.InheritCollectionPermissions = collectionpermission, org.dspace.ctask.general.DeleteReplicationItems = deletereplicationitems]

It looks correct.  Do you have any ideas what else could cause not resolved issue?

Thanks,
Enes

[Tim Donohue]

Hi Enes,

Huh, that is weird.  That configuration / output looks perfectly reasonable to me, and I see it includes an entry named "requiredmetadata" which is the exact task that threw the "Task: requiredmetadata not resolved" error.   So, that's very odd to me... DSpace seems to see you have a configuraiton for "requiredmetadata" but cannot find it when it tries to run it.

To be honest, that's a bit baffling to me.

One thing you could try is to temporarily replace your configurations with the *default* configurations from DSpace 6.x.  For example, copy the curate.cfg for DSpace 6.x from here: https://github.com/DSpace/DSpace/blob/dspace-6_x/dspace/config/modules/curate.cfg   And then double check that you have this "include" line in your dspace.cfg: https://github.com/DSpace/DSpace/blob/dspace-6_x/dspace/config/dspace.cfg#L2005

After you've temporarily changed your configs, try running the "requiredmetadata" task again.  If it works, then that implies one of the other custom configs in your curate.cfg might be to blame...you can add them back in one-by-one to see which one causes the failure.  If it still doesn't work, then it's not your curate.cfg file at all.

- Tim

[enes...@yahoo.com]

Hi Tim,

I have tried to copy the original curate.cfg, but it is the same outcome.  I do not thing that it is the problem with curate.cfg.  The strange thing is that curation tasks have worked on this server for several months and tnen stopped working.

Regards,
Enes

[Tim Donohue]

Hi Enes,

Sorry, I've run out of ideas here.  As you aren't able to reproduce this on another server, and I cannot reproduce this on my end (or on demo.dspace.org), it seems to me it's likely to be one of the following (but I don't know which one):

1. A configuration problem in either curate.cfg or dspace.cfg

2. A code customization issue -- where some Java code was customized in the DSpace API, and it's breaking how the Curation code normally functions

3. Possibly (though less likely) a DSpace upgrade issue, where some old code is still "in place" (like an old JAR file or something) that is conflicting with the current code.

That's my best guess here. If we could find a way to reproduce this issue elsewhere, we might be able to narrow down the issue better.  But, without reproducibility, I'm mostly just guessing at things that could be different on this server.  

You also might want to brainstorm what changed about this server recently, as it sounds like something must have changed in order for it to stop working suddenly... e.g. if you upgraded recently, maybe the upgrade messed something up.  If you changed a configuration recently, maybe try undoing recent config changes to see if they had an effect.  If you implemented some custom code recently, maybe try undoing that to see if that could be the cause.

Sorry I cannot be of more help.  If you figure out more about what could be going on, please do let us know on this list (especially if you start to narrow down the cause). It really could help others to debug issues that they see that might be similar in the future.

Good luck,

Tim

[enes...@yahoo.com]

Hi Tim,

I was able to solve this issue. It was tied to input forms. Our instance is multilingual (4 languages) . It looks like that someone was changing translation versions of the input form, while the main version was unchanged or there was mismatch. The main input form form was syntactically correct. At the same time, around 100GB of data was imported into system. When I have replaced the main input form with earlier version, everything started to function again.

Thanks again for your help.

Regards,

Enes