MakeDataCounts - check your JWT
James Myers
FYI: At QDR, we just discovered that the JWT we have from DataCite that allows submitting usage reports expired after one year and counter-processor has been dutifully noting a 401 response from DataCite in the daily log file. Since this doesn’t affect counts getting uploaded into Dataverse we/I hadn’t noticed.
If you have enabled Make Data Count on your Dataverse instance and have configured counter-processor to upload_to_hub: True and are running the counter_daily.sh script, you may want to check the JWT you have in the ./config/secrets.yaml file and/or check the datacite_response_body.txt log file (in /tmp with the defaults) to verify that you have a valid JWT and that the upload calls are succeeding. A simple way to check the expiration date is to cut/paste your JWT into https://jwt.io/ . (Note this site indicates it processes your secret token only in your local browser – if you use other sites to check be sure you trust them). In our case the Payload includes the following, which, when you hover over it shows a human readable expiration date:
My guess is that reprocessing will be needed to send data for affected months to DataCite – still investigating that. (The Dataverse mdc logs are persistent, but counter-processor overwrites its reports each day – not sure if counter-processor can regenerate those without reprocessing the underlying logs.) I also will try to look into ways to make this problem more visible (has anyone added monitoring for this already?. I’d be happy to coordinate and/or hand-off either of those to others if you’re interested!)
In the meantime, I thought I’d get the word out so those using MakeDataCount can check/request fresh JWTs as needed.
-- Jim
Philipp at UiT
James Myers
DataCite appears to provide JWT’s that are good for one year. At QDR, we’ve put a reminder on the calendar to request a new one yearly. If you have the ability to get one for a longer duration – it’s basically a trade-off between convenience and security.
W.r.t. catching up at DataCite, what I ended up doing at QDR is combining this with an upgrade to the latest counter-processor version (0.1.04) and then re-running the months when our JWT was expired (the same process as when running back months when you initially start using counter-processor). I did not see any easy way to trigger a resend to DataCite without re-running those months (letting c-p process the Dataverse make-data-counts log files again), but presumably c-p’s internal db entries have the required info (there’s just no way to make the separate DataCite API call for those months without digging in the c-p code).
For other sites, I think this should work as well. I’ve verified that Dataverse isn’t affected by the reprocessing (if you run the counter_daily.sh script the last API call in it will rewrite the count entries for that month in Dataverse, but it won’t cause double–counting). (Note: the newer c-p uses gzip encoding when it sends to DataCite which is what they recommend for larger transfers. I did check the reports that were sent from QDR using the DataCite reports API and using standard tools to decode and unzip the report contents to confirm it was the same as the local report/what is sent to Dataverse) (e.g. echo <base64 block> | base64 --decode > temp.json.gz and gunzip temp.json)
If you don’t upgrade to c-p 0.1.04 at the same time, you will probably have to adjust c-p’s state (in the ./state subdir) before it will reprocess the months you need instead of assuming those are already done (not completely sure – I did not try this).
Hope that helps,
-- Jim
--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
dataverse-commu...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/dataverse-community/d574dd4a-f678-4bea-b639-3a482b7be5dfn%40googlegroups.com.