Restricted file download fails for larger files if Terms of Access is specified

[Zhang Shuxian]

Dear Dataverse Community,

 

We have encountered issues when trying to download restricted files with "Terms of Access" specified.

 

1st Issue

 

When "Terms of Access" is specified, and a user is granted permission to download restricted files. Example dataset, https://doi.org/10.21979/N9/BRIOZ0.

 

• If we start by downloading very small files (e.g., the 625B, 1.3KB files), the downloads are successful. And then we can proceed to download larger files successfully.

 

• If we start by downloading larger files (e.g., the 5.2KB, 5.8KB files or anything larger), the download never starts after we click to accept the “Term of Access”.

 

 

For datasets with restricted files and no "Terms of Access" specified (e.g., https://doi.org/10.21979/N9/3BMCMW), a user with granted permission can download files of several gigabytes large, without any issue.

 

The same thing happens to superusers as well.

 

 

2nd Issue

 

For datasets with "Terms of Access" is specified (e.g., https://doi.org/10.21979/N9/BRIOZ0), the pop-up message sometimes become very weird, and we cannot predict when this happens.

 

Most of the time, the correct pop-up is shown (screenshotted below) when we click to download a restricted file.

 

 

 

But sometimes, a strange page is shown (looks like a guestbook, but there is no guestbook assigned to this dataset).

 

 

 

Has anyone encountered a similar issue?

 

We are currently at Version 6.1, but we are not sure when the issues start.

 

Thanks very much.

 

Warm regards,

Shuxian from NTU Library

---

CONFIDENTIALITY: This email is intended solely for the person(s) named and may be confidential and/or privileged. If you are not the intended recipient, please delete it, notify us and do not copy, use, or disclose its contents.
Towards a sustainable earth: Print only when necessary. Thank you.

[Julian Gautier]

Hi Shuxian,

Thanks for describing these issues!

I couldn't help with the first one you described, except to comment that I can't imagine how the problem is related to those super small file sizes.

But for the second issue you described, I can say that it might be related to a series of guestbook bugs that were fixed in later versions of Dataverse, like the bug described in https://github.com/IQSS/dataverse.harvard.edu/issues/247#issuecomment-2627541100 that informed changes described in the pull request https://github.com/IQSS/dataverse/pull/11203. The changes in that PR were included in the v6.6 release of Dataverse. So I'm wondering if upgrading the version of Dataverse that your repository is using would help.

Or maybe you could test things on a later version of Dataverse. Our testing site, Demo Dataverse (https://demo.dataverse.org), is using v6.10 and you would be welcome to use it for testing if you'd like.

Hope this helps!

Julian

Julian Gautier (he/him)

Product Research Specialist, IQSS

Interested in helping test Dataverse? Sign up for user experience research

[Philip Durbin]

Hi Shuxian,

I'm afraid I don't have any particular insight on these. I don't remember any related GitHub issues.

To reiterate what Julian said, perhaps upgrading will help? And yes, if you can reproduce any of this on our demo site, please let us know!

Thanks,

Phil

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/dataverse-community/8148a8ba-4b5b-4561-92ba-bb704613f9c5n%40googlegroups.com.

--

Philip Durbin
Software Developer for http://dataverse.org
http://www.iq.harvard.edu/people/philip-durbin

[Shuxian Zhang]

Hi Julian and Philip,

Thanks a lot for your responses.

I have tried the Demo Dataverse (https://demo.dataverse.org), and both issues were not replicated. 

The two issues currently affect all our restricted datasets with "Terms of Access" specified.

 Is the bug fixed in v6.6 and later? We likely will not have enough resources to upgrade our application to v6.6 within this year. 

I'm wondering if there is other solutions besides upgrading.

Best regard,

Shuxian from NTU Library

[Julian Gautier]

Hey Shuxian,

Yeah the PR I mentioned made changes to v6.6 of Dataverse, so I'd guess that at least for the second issue you mentioned, the bug is fixed for v6.6 and later.

I'm still not sure about that first issue with the file sizes.

About other solutions, it's tough for me to say or test other solutions since I don't have access to a repository that's using a version that we released before v6.6 (and don't want to abuse the other Dataverse repositories that are on v6.1 😅).

I know you wrote that no datasets have guestbooks applied to them, but for the collections that those datasets are in, what's the guestbook mode set to?  If it's set to "Guestbook Entry at Access Request", do the bugs still exist if you change the guestbook mode to "Guestbook Entry at Download"?

[Shuxian Zhang]

Hi Julian,

Thanks a lot for your advice.

We have tried setting "Guestbook Entry at Download", but a guestbook still pops up when users click "Request Access" to a restricted file (and the dataset has no guestbook assigned).

Sad... 

Thank you all for your kind attention and advice.

I guess we may have to wait until we upgrade to higher version.

Thanks again.

Best regards,

Shuxian from NTU Library

[Julian Gautier]

Hi Shuxian,

Thanks for checking!

I was worried if folks running other Dataverse repositories were trying to resolve this bug, too, and I just realized that I could reproduce this bug if I could request access to a restricted file in a dataset without a guestbook and published in another repository using version 6.1 of the Dataverse software.

One of the three known repositories using Dataverse v6.1 is Centro Brasileiro de Pesquisas Físicas (CBPF). Luckily they let anyone create an account, so I was able to experience what happens when I request access to any of the restricted files in the dataset at https://doi.org/10.82037/CBPF/INXKOD, which has no guestbook. All files are restricted and they were all downloaded once. In the Dataset Terms popup, below the CC-BY license, I can see the default guestbook questions, like Name and Email. So the bug exists there, too.

Here's what that popup looks like:

I don't think the folks managing the CBPF repository have mentioned this bug, and I wonder if that's because it doesn't seem so bad. I didn't have to do anything extra because all the fields are prefilled from my account, all fields are optional, and it might not be obvious to the requester that this is a "guestbook".

It's also possible that the dataset's 19 downloads are from someone who doesn't have to request access, like the dataset depositor or repository staff, so whoever downloaded the files might never have seen this  Dataset Terms popup.

In any case, until you can upgrade to a higher version, I hope the bug isn't too much of a headache.

Sincerely,

Julian

[Shuxian Zhang]

Hi Julian, 

Thanks a lot for testing this in other repositories using v6.1.

It's a relief to know that my repository isn't the only one with this issue.

And yes, you're right, this guestbook popup is not too much of a trouble for our repository users.

Thanks again. :)

Best regards,

Shuxian from NTU Library