Dataverse 6.2 - problem with handles pid (Handle.net)

[Volodymyr Kushnarenko]

Dear community and dear Dataverse developers,

I have a problem with configuration of Dataverse 6.2 with Handle.net.

With the new approach with multiple PID providers (with options "dataverse.pid.providers" and "dataverse.pid.default-provider") I can create a new dataset via GUI (new handle can be properly generated) but in server.log I see many exceptions that connection to the handle server was wrong, so physically the new handle was not registered:

[2024-06-19T15:16:29.234+0200] [Payara 6.2024.5] [WARNING] [] [edu.harvard.iq.dataverse.pidproviders.handle.HandlePidProvider] [tid: _ThreadID=133 _ThreadName=http-thread-pool::jk-connector(5)] [timeMillis: 1718802989234] [levelValue: 900] [[
  Caught exception trying to process lookup request
HandleException (SERVICE_NOT_FOUND) Unable to find service for prefix 0.NA/ <XX.XXXXX-Prefix> ; prefix resolution response: Error(301): SERVER NOT RESPONSIBLE FOR HANDLE: That prefix doesn't live here
        at net.handle.hdllib.HandleResolver.tryAuthGlobalServiceLookupAndThrowExceptionOnFailure(HandleResolver.java:901)
        at net.handle.hdllib.HandleResolver.getServiceInfo(HandleResolver.java:873)
        at net.handle.hdllib.HandleResolver.findLocalSites(HandleResolver.java:910)
        at net.handle.hdllib.HandleResolver.processRequestGlobally(HandleResolver.java:617)
        at net.handle.hdllib.HandleResolver.processRequest(HandleResolver.java:587)
        at net.handle.hdllib.HandleResolver.processRequest(HandleResolver.java:595)
        at edu.harvard.iq.dataverse.pidproviders.handle.HandlePidProvider.isHandleRegistered(HandlePidProvider.java:210)
        at edu.harvard.iq.dataverse.pidproviders.handle.HandlePidProvider.alreadyRegistered(HandlePidProvider.java:332)

...

What is also interesting, on the logs of our handle server we see requests from Dataverse instance, but looks like they are empty or not complete:

-> UDP requests are only of 2 types:

<IP-Address> UDP:HDL(2.11) "2024-06-19 10:30:53.331+0200" 1 301 3ms  0.NA/<XX.XXXXX-Prefix>
<IP-Address> UDP:HDL(2.11) "2024-06-19 10:30:53.333+0200" 1 301 5ms  0.0/0.0

-> TCP requests have only 1 type:

<IP-Address> TCP:HDL(2.11) "2024-06-19 10:30:53.478+0200" 1 301 3ms  0.0/0.0

When we create new handles via other client (not from the Dataverse) we see something like this:

<IP-Address>  HTTP:HDLApi "2024-06-19 09:17:46.791+0200" 105 1 26ms adm=300:0.NA/<XX.XXXXX-Prefix> <XX.XXXXX-Prefix>

So, can be that Dataverse sends wrong/empty requests?

My configuration in domain.xml:

        <jvm-options>-Ddataverse.pid.handle1.handlenet.key.path=<PATH-TO-FILE>/admpriv.bin</jvm-options>
        <jvm-options>-Ddataverse.pid.handle1.handlenet.key.passphrase=${ALIAS=xxxxx}</jvm-options>
        <jvm-options>-Ddataverse.pid.handle1.handlenet.index=300</jvm-options>
        <jvm-options>-Ddataverse.pid.handle1.type=hdl</jvm-options>
        <jvm-options>-Ddataverse.pid.handle1.label=Handle label</jvm-options>
        <jvm-options>-Ddataverse.pid.handle1.authority=XX.XXXXX</jvm-options>
        <jvm-options>-Ddataverse.pid.handle1.shoulder=handle-pid/</jvm-options>
        <jvm-options>-Ddataverse.pid.handle1.identifier-generation-style=storedProcGenerated</jvm-options>
        <jvm-options>-Ddataverse.pid.handle1.datafile-pid-format=DEPENDENT</jvm-options>
        <jvm-options>-Ddataverse.pid.handle1.handlenet.auth-handle="0.NA/XX.XXXXX"</jvm-options>
        <jvm-options>-Ddataverse.pid.handle1.handlenet.independent-service=false</jvm-options>
        <jvm-options>-Ddataverse.pid.fake.type=FAKE</jvm-options>
        <jvm-options>-Ddataverse.pid.fake.label=Fake DOI Provider</jvm-options>
        <jvm-options>-Ddataverse.pid.fake.authority=10.5072</jvm-options>
        <jvm-options>-Ddataverse.pid.fake.shoulder=fake-pid-</jvm-options>
        <jvm-options>-Ddataverse.pid.fake.identifier-generation-style=storedProcGenerated</jvm-options>
        <jvm-options>-Ddataverse.pid.providers=fake,handle1</jvm-options>
        <jvm-options>-Ddataverse.pid.default-provider=handle1</jvm-options>

Basing on this I have 2 questions:

1) Did anybody have similar problem with Handle.net and Dataverse 6.2? I thought maybe admpriv.bin has wrong permissions, but looks like it is readable and credentials are correct also (because changing credentials I get another exceptions).

2) If approach with many pid providers has currently some problems, can I still use the previous approach with only 1 provider? I have tried to configure it basing on the documentation, but I also have some problems - looks like currently the variables "dataverse.pid.providers" and "dataverse.pid.default-provider" are must have! If I remove "dataverse.pid.providers" payara can not start, you see exceptions in log. If "dataverse.pid.default-provider" is missing, then you see exceptions when you try to create new dataset. How can look like a configuration for only-one pid provider then? Because I tried with different combinations of jvm-options and database variables and I did not find a working one, I always got an exception.

I thank you very much in advance for your quick response.

All the best,

Vladimir

[James Myers]

Vladimir,

I don’t see anything obvious looking at your settings and digging into the code a bit. In general, the switch to supporting multiple PID providers didn’t change the code internal to a provider itself, so any bugs are likely to be related to changes between the old and new settings. If you had this running in an earlier version, it might be helpful if you can share the settings you were using then.

 

To try the old style of settings, you’re right that you need to have the providers and default-provider settings. The simplest solution is probably to leave the fake1 provider in place, list it as the only entry in providers but then set default-provider to “legacy”. You’ll need to remove/change handle1’s definition so that it doesn’t share the same protocol/authority/shoulder as your old settings (the new settings have precedence when they have the same protocol/auth/shoulder). Alternately, instead of making “legacy” the default, you can leave that as “fake1” and, in the UI, log in as a superuser, go to your root collection, click the Edit/General Information menu item and look for the PID Provider Option – you should be able to select fake1 or “legacy” there and any new datasets will use the provider you choose.

 

Hopefully this at least gives you a work-around. If not, just seeing your old working settings might give me a better chance to dig into the code to see what’s different now. If it turns out there’s a bug in the code or documentation, please create an issue so we can resolve it for 6.3.

 

-- Jim

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/78e10480-dca1-4f33-bf97-1e973afd463dn%40googlegroups.com.

[Volodymyr Kushnarenko]

Dear Jim,

many thanks for your quick reply!! After investigation we found out, that problem was because of firewall configuration of our handle server. So, Dataverse itself works correct! The new approach with multiple PID providers (see config above in my first message) works correct!

I have also tried an approach with the "legacy" provider. Configuration can be e.g. like this:

        <jvm-options>-Ddataverse.pid.fake.type=FAKE</jvm-options>
        <jvm-options>-Ddataverse.pid.fake.label=Fake DOI Provider</jvm-options>
        <jvm-options>-Ddataverse.pid.fake.authority=10.5072</jvm-options>
        <jvm-options>-Ddataverse.pid.fake.shoulder=fake-pid-</jvm-options>
        <jvm-options>-Ddataverse.pid.fake.identifier-generation-style=storedProcGenerated</jvm-options>

        <jvm-options>-Ddataverse.pid.providers=fake</jvm-options>
        <jvm-options>-Ddataverse.pid.default-provider=legacy</jvm-options>
        <jvm-options>-Ddataverse.pid.handlenet.key.path= <PATH-TO-FILE>/admpriv.bin </jvm-options>

        <jvm-options>-Ddataverse.pid.handlenet.key.passphrase= ${ALIAS=xxxxx} </jvm-options>       

        <jvm-options>-Ddataverse.pid.handlenet.index=300</jvm-options>

with all needed additional database settings for Handles from here: https://guides.dataverse.org/en/6.2/installation/config.html#configuring-your-dataverse-installation-for-a-single-handle-provider

Also different combinations of "fake" and "legacy" strings for "providers" and "default-provider" work correct as you wrote. The only thing what was a problem for me - for version 5.x I deleted the database setting ":DoiProvider" as was proposed here for the Handle configuration [https://groups.google.com/g/dataverse-community/c/GdWxd26bJDo/m/xF2fQ_74AAAJ], and for 6.2 it must be not-null because of this code [https://github.com/IQSS/dataverse/blob/3dccdb7506bf642a4d331ae00ca3f52cbbd4baf1/src/main/java/edu/harvard/iq/dataverse/pidproviders/PidProviderFactoryBean.java#L147] (see next "if" there), so I recreated it again.

So, if for 6.2+ versions you plan to keep "legacy" support, then maybe some mention of this "legacy" parameter and importance of ":DoiProvider" in the documentation would be nice to have. Please tell me shortly, if I should create an issue for that.

Again many thanks for your work!

Vladimir

[James Myers]

Vladimir,

Thanks for checking the legacy config in detail. It would be great if you can submit an issue, and a PR if you like – as you say it looks like an easy fix in that the code should allow a null :DoiProvider unless the protocol is doi (right now it has to be non-null for all cases.)

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/8ba97d2d-6999-4a34-bd0a-9877953ca391n%40googlegroups.com.

[Volodymyr Kushnarenko]

Dear Jim,

Many thanks too, the issue is created: https://github.com/IQSS/dataverse/issues/10658

Vladimir