Problems with Harvested Records after upgrade to Dataverse v5.8

47 views
Skip to first unread message

wongka...@gmail.com

unread,
Nov 15, 2022, 1:11:55 AM11/15/22
to Dataverse Users Community
Dear Colleagues, 

We have an arrangement where NTU Dataverse ( https://researchdata.ntu.edu.sg ) harvests records from NIE Dataverse ( https://researchdata.nie.edu.sg ). 

The above is working fine until NIE upgrades the Dataverse version from v5.4 to v5.8.

Upon some investigation, NTU found via SQL that the records harvested from NIE have "versionnumber" as null even though "versionstate" is "RELEASED" in the table "datasetversion". 

However, when NIE check the same set of records in NIE instance via SQL, the "versionnumber" is not null when "versionstate" is "RELEASED" in the table "datasetversion".

Apparently, the harvesting via OAI did not capture the "versionnumber" correctly.

Prior to the Dataverse upgrade from v5.4 to v5.8, this was not an issue for us. 

Hence, we are wondering if the community has encountered anything similar please? 

Would love to hear some suggestions on things we can tweak / check to resolve this issue. 

Thank you. 

Regards,
Jacky

Philip Durbin

unread,
Nov 28, 2022, 3:43:48 PM11/28/22
to dataverse...@googlegroups.com
Hi Jacky,

Just to add one more detail, I assume that NTU has been on 5.10.1 the whole time (like it is now). Only NIE was upgraded (from 5.4 to 5.8, like you said).

I'm not aware of any bugs in this area but we're investigating harvesting bugs in general right now so maybe we can take a look at this. Can you please open an issue at https://github.com/IQSS/dataverse/issues ?

To help troubleshoot, it might be nice to have the output of the client config. I'll show an example below.

Thanks,

Phil

p.s. Client config:


{
  "status": "OK",
  "data": {
    "harvestingClients": [
      {
        "nickName": "demo",
        "dataverseAlias": "root",
        "type": "oai",
        "harvestUrl": "https://demo.dataverse.org/oai",
        "archiveUrl": "https://demo.dataverse.org",
        "archiveDescription": "This Dataset is harvested from our partners. Clicking the link will take you directly to the archival source of the data.",
        "metadataFormat": "oai_dc",
        "set": "geospatial_metadata",
        "schedule": "none",
        "status": "inActive",
        "lastHarvest": "Mon Nov 28 15:33:06 EST 2022",
        "lastResult": "SUCCESS",
        "lastSuccessful": "Mon Nov 28 15:33:06 EST 2022",
        "lastNonEmpty": "N/A",
        "lastDatasetsHarvested": "N/A",
        "lastDatasetsDeleted": "N/A",
        "lastDatasetsFailed": "N/A"
      }
    ]
  }
}

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/ce1ddad7-1435-44fe-958e-d2a4c80ce073n%40googlegroups.com.


--

wongka...@gmail.com

unread,
Nov 29, 2022, 12:56:35 AM11/29/22
to Dataverse Users Community
Dear Philip, 

Yes. Your understanding is correct. Ok, we shall create the open issue at https://github.com/IQSS/dataverse/issues

Thank you. 

Regards,
Jacky

Philip Durbin

unread,
Jan 5, 2023, 7:55:13 AMJan 5
to dataverse...@googlegroups.com
Reply all
Reply to author
Forward
0 new messages