--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/f0663875-67f6-477e-b05b-117e9cbe040f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Hi Thanh Thanh,Yes, unfortunately I've seen this on test environments (my laptop and two test servers) and wrote a little bit about it at https://github.com/IQSS/dataverse/issues/5393#issuecomment-447441742For my laptop and one of my test servers ("phoenix", see https://github.com/IQSS/dataverse/issues/5409 ) I've switched over to a new "fake" persistent ID provider that's only available as of Dataverse 4.10: http://guides.dataverse.org/en/4.10/developers/dev-environment.html#configure-your-development-environment-for-publishingI'll probably move my other test server over to the fake PID provider as well but it's only a solution for test environments, not for production. I didn't know production installations were affected! Can you please create a GitHub issue about this?Thanks,Phil
On Thu, Jan 3, 2019 at 2:49 AM Thanh Thanh Le <lethithan...@gmail.com> wrote:
Hello everyone,--Firstly, Happy New Year to everybody ! ^^I want to ask, if anyone of you have the problem publishing a dataset with file ?Recently, we can not publish a dataset if a file is added : the operation is not done without any particular error (IHM or logs).We have dataverse 4.9.2 on test env and 4.9.1 on production, connecting to Datacite. And the problem occurs in the 2 environnements.Thanks in advance,Best regards,Thanh Thanh
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/f0663875-67f6-477e-b05b-117e9cbe040f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/f0663875-67f6-477e-b05b-117e9cbe040f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--Philip Durbin
Software Developer for http://dataverse.org
http://www.iq.harvard.edu/people/philip-durbin
--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/98651857-3928-4300-8b5e-fc43bab7f5a7%40googlegroups.com.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/f0663875-67f6-477e-b05b-117e9cbe040f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--Philip Durbin
Software Developer for http://dataverse.org
http://www.iq.harvard.edu/people/philip-durbin
--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/f0663875-67f6-477e-b05b-117e9cbe040f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--Philip Durbin
Software Developer for http://dataverse.org
http://www.iq.harvard.edu/people/philip-durbin
--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse-community@googlegroups.com.
All,
An update on this issue based on some community debugging…
It appears that DataCite made a change during their maintenance period that exposed a minor bug in Dataverse.
What’s Happening/Symptoms: When checking whether newly minted IDs for files already exist at the provider, Dataverse was accidentally sending a query about “null” rather than the new ID. Previously DataCite had responded (we assume) with a 404/not found response and publication proceeded (it would have missed if the DOI was already registered at DataCite but was not in the Dataverse database already).
After their update, they started sending a 200 response with the first page of a list of 16M DOIs. That is interpreted by Dataverse as meaning the new file ID already exists and it enters a loop to keep generating new IDs. That ~infinite loop either fails with a timeout after several minutes with no log message, or causes DataCite to bog down and respond with a 502 gateway timeout – which also causes publication to fail but does show an exception in the log.
Who/what’s affected:
Assuming this is the only issue, it only affects installations configured to use file IDs and it only affects the publication of versions including new files (v1.0 or later versions where a file has been added), which probably helps explain why it appears intermittent.
Fix/work-around:
In https://github.com/IQSS/dataverse/issues/5427 and the associated https://github.com/IQSS/dataverse/pull/5428, there’s a fix that Dataverse should be able to roll into a 4.10.1 release – timing TBD @ IQSS. The fix sends the real DOI to DataCite (or other provider), fixing the bug, and allowing publication to succeed.
In the meantime, and for 4.9.x versions, the only work-around there is so far is to disable file DOIs until one updates to a 4.10.x version with this fix. (The same /api/admin/registerDataFileAll used to convert to file DOIs to start should pick up any from publications while file DOIs are turned off. )
(I plan to report this to DataCite, but it is only partly their issue, so I don’t expect anything quick from their end that would fix things for 4.9.x installations…)
Other issues:
If anyone is seeing a publishing problem that doesn’t fit this picture, please create a new issue.
-- Jim
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/f0663875-67f6-477e-b05b-117e9cbe040f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--Philip Durbin
Software Developer for http://dataverse.org
http://www.iq.harvard.edu/people/philip-durbin
--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/98651857-3928-4300-8b5e-fc43bab7f5a7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to
dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/a3d789d4-af46-438f-b7e0-87e5687cdb04%40googlegroups.com.
Hi everyone,
When we got back from break yesterday, we started investigating some reported issues with publishing on Harvard's Dataverse. Some Datasets publish but others do not.
We think this is limited to the Harvard installation, but this thread leads me to believe other installations using DataCite may be seeing issues as well. I'll update this as we uncover more information.
- Danny
On Thursday, January 3, 2019 at 9:29:19 AM UTC-5, Donald Sizemore II wrote:
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/a3d789d4-af46-438f-b7e0-87e5687cdb04%40googlegroups.com.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/f0663875-67f6-477e-b05b-117e9cbe040f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Philip Durbin
Software Developer for http://dataverse.org
http://www.iq.harvard.edu/people/philip-durbin
--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/98651857-3928-4300-8b5e-fc43bab7f5a7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/f0663875-67f6-477e-b05b-117e9cbe040f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Philip Durbin
Software Developer for http://dataverse.org
http://www.iq.harvard.edu/people/philip-durbin
--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/98651857-3928-4300-8b5e-fc43bab7f5a7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/a3d789d4-af46-438f-b7e0-87e5687cdb04%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/9a4163ee-e3a0-497e-b741-4613c4e1650f%40googlegroups.com.
For those not watching the IQSS/Dataverse list:
DataCite has fixed the issue on their end. I’ve verified that this allows QDR’s v4.9.4 installation to publish again. With that, the only effect of the bug in Dataverse (fixed with v4.10.1) is that Dataverse would not catch the situation where the DOI being created for a file during publication duplicates one already at DataCite but not created by Dataverse. This can only occur if you have multiple Dataverse instances using the same authority/shoulder or have some other system creating DOIs with that authority/shoulder. (Dataverse will catch the case where the DOI matches one already created in that Dataverse and will generate a different one).
-- Jim
From: qqm...@hotmail.com
Sent: Thursday, January 03, 2019 2:52 PM
To: Dataverse Users Community
Subject: RE: [Dataverse-Users] Recently can not publish dataset with file (with Datacite)
All,
An update on this issue based on some community debugging…
It appears that DataCite made a change during their maintenance period that exposed a minor bug in Dataverse.
What’s Happening/Symptoms: When checking whether newly minted IDs for files already exist at the provider, Dataverse was accidentally sending a query about “null” rather than the new ID. Previously DataCite had responded (we assume) with a 404/not found response and publication proceeded (it would have missed if the DOI was already registered at DataCite but was not in the Dataverse database already).
After their update, they started sending a 200 response with the first page of a list of 16M DOIs. That is interpreted by Dataverse as meaning the new file ID already exists and it enters a loop to keep generating new IDs. That ~infinite loop either fails with a timeout after several minutes with no log message, or causes DataCite to bog down and respond with a 502 gateway timeout – which also causes publication to fail but does show an exception in the log.
Who/what’s affected:
Assuming this is the only issue, it only affects installations configured to use file IDs and it only affects the publication of versions including new files (v1.0 or later versions where a file has been added), which probably helps explain why it appears intermittent.
Fix/work-around:
In https://github.com/IQSS/dataverse/issues/5427 and the associated https://github.com/IQSS/dataverse/pull/5428, there’s a fix that Dataverse should be able to roll into a 4.10.1 release – timing TBD @ IQSS. The fix sends the real DOI to DataCite (or other provider), fixing the bug, and allowing publication to succeed.
In the meantime, and for 4.9.x versions, the only work-around there is so far is to disable file DOIs until one updates to a 4.10.x version with this fix. (The same /api/admin/registerDataFileAll used to convert to file DOIs to start should pick up any from publications while file DOIs are turned off. )
(I plan to report this to DataCite, but it is only partly their issue, so I don’t expect anything quick from their end that would fix things for 4.9.x installations…)
Other issues:
If anyone is seeing a publishing problem that doesn’t fit this picture, please create a new issue.
-- Jim
From: dataverse...@googlegroups.com [mailto:dataverse...@googlegroups.com]
On Behalf Of danny...@g.harvard.edu
Sent: Thursday, January 03, 2019 10:03 AM
To: Dataverse Users Community
Subject: Re: [Dataverse-Users] Recently can not publish dataset with file (with Datacite)
Hi everyone,
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/f0663875-67f6-477e-b05b-117e9cbe040f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--Philip Durbin
Software Developer for http://dataverse.org
http://www.iq.harvard.edu/people/philip-durbin
--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/98651857-3928-4300-8b5e-fc43bab7f5a7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to
dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/a3d789d4-af46-438f-b7e0-87e5687cdb04%40googlegroups.com.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/f0663875-67f6-477e-b05b-117e9cbe040f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Philip Durbin
Software Developer for http://dataverse.org
http://www.iq.harvard.edu/people/philip-durbin
--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/98651857-3928-4300-8b5e-fc43bab7f5a7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/a3d789d4-af46-438f-b7e0-87e5687cdb04%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/9a4163ee-e3a0-497e-b741-4613c4e1650f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/CABbxx8EyYsA-Gm3poQwQDHD4-8DzwRCqpu83U2xeeX2T2GezRQ%40mail.gmail.com.