Is there an easy(-ish) way to set data files to 'restricted' by default

43 views
Skip to first unread message

MM - ADA

unread,
Sep 9, 2019, 9:58:21 PM9/9/19
to Dataverse Users Community
Hi - 99% of our data files are restricted. 

Is there an easy way to set the default for uploaded files to be restricted so 
1) we don't have to set it manually to restricted for every single file 
2) we know the default is restricted so we don't inadvertently forget to set it to restricted

This will require un-restricting any files that aren't restricted, of course, but that is preferable than potentially not restricting a file that needs to be.

It would be nice to have an 'Upload restricted files' button that would upload the files and automatically set them to restricted
and
an 'Upload files' button for non-restricted files

Thanks for feedback/advice/solutions.

M.




Philip Durbin

unread,
Sep 9, 2019, 10:30:56 PM9/9/19
to dataverse...@googlegroups.com
No, there isn't an easy way to have "restricted" be the default for files.

This is "installation persona" territory... Some installations of Dataverse have 100% open access data (not about human subjects) and others (like yours) have almost entirely restricted data.

":PublicInstall" is a configuration setting that removes the ability to restrict data: http://guides.dataverse.org/en/4.16/installation/config.html#publicinstall ... It sounds like you want the opposite, except for your 1% of files that can be public. :)

So many there should be a new installation-wide setting? ":RestrictedInstall"? Or some other solution? It'll take some thought and development.

You *can* set files to be restricted on upload via API ( http://guides.dataverse.org/en/4.16/api/getting-started.html#uploading-files ) so you *could* write your own client or tool that always sets "restrict:true" but I assume you'd rather have a setting that affects the GUI.

I hope this helps,

Phil

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/0b82407b-163a-4d7e-84c4-426409bb7c10%40googlegroups.com.


--

MM - ADA

unread,
Sep 9, 2019, 10:41:23 PM9/9/19
to Dataverse Users Community
Thanks Phil - I thought as much but wanted to confirm.

I am aware of the API capability but we would like it for the UI as well. Being able to restrict on upload would be ideal.

I also thought of setting the default in the database tables that indicate a file is restricted to 't' but messing around with the database itself isn't desirable unless absolutely necessary.

=> ":PublicInstall" is a configuration setting that removes the ability to restrict data: http://guides.dataverse.org/en/4.16/installation/config.html#publicinstall ... It sounds like you want the opposite, except for your 1% of files that can be public. :)  
         => This is an accurate assessment!

M.

Philip Durbin

unread,
Sep 10, 2019, 1:40:45 AM9/10/19
to dataverse...@googlegroups.com
Yeah, if you're tempted to hack on the database directly I'd recommend writing a script instead that does something like this:

- Iterate through all datasets, finding files that are restricted (export formats such as "JSON" or DDI will tell you).
- For all unrestricted files, restrict them with http://guides.dataverse.org/en/4.16/api/native-api.html#restrict-files

Or you could instead build a report of unrestricted files. Maybe you could maintain a skip list of files that are supposed to be unrestricted.

Just a thought,

Phil

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/38a50cf5-20c8-4694-a818-6afa6d477800%40googlegroups.com.

Stefan Kasberger

unread,
Sep 10, 2019, 9:54:34 AM9/10/19
to Dataverse Users Community
Hi,

we at AUSSDA would be happy about that option too, cause from a ethics/copyright/gdpr point of view, to open a restricted file is a bigger problem, than to restrict an open one. Sounds reasonable to set the prefered option somewhere in the backend (radio button).

Regards, Stefan

Am Dienstag, 10. September 2019 04:30:56 UTC+2 schrieb Philip Durbin:
No, there isn't an easy way to have "restricted" be the default for files.

This is "installation persona" territory... Some installations of Dataverse have 100% open access data (not about human subjects) and others (like yours) have almost entirely restricted data.

":PublicInstall" is a configuration setting that removes the ability to restrict data: http://guides.dataverse.org/en/4.16/installation/config.html#publicinstall ... It sounds like you want the opposite, except for your 1% of files that can be public. :)

So many there should be a new installation-wide setting? ":RestrictedInstall"? Or some other solution? It'll take some thought and development.

You *can* set files to be restricted on upload via API ( http://guides.dataverse.org/en/4.16/api/getting-started.html#uploading-files ) so you *could* write your own client or tool that always sets "restrict:true" but I assume you'd rather have a setting that affects the GUI.

I hope this helps,

Phil

On Mon, Sep 9, 2019 at 9:58 PM MM - ADA <Marina...@anu.edu.au> wrote:
Hi - 99% of our data files are restricted. 

Is there an easy way to set the default for uploaded files to be restricted so 
1) we don't have to set it manually to restricted for every single file 
2) we know the default is restricted so we don't inadvertently forget to set it to restricted

This will require un-restricting any files that aren't restricted, of course, but that is preferable than potentially not restricting a file that needs to be.

It would be nice to have an 'Upload restricted files' button that would upload the files and automatically set them to restricted
and
an 'Upload files' button for non-restricted files

Thanks for feedback/advice/solutions.

M.




--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.

Philip Durbin

unread,
Sep 10, 2019, 10:41:48 AM9/10/19
to dataverse...@googlegroups.com
Ok. Somebody, please create an issue at https://github.com/IQSS/dataverse/issues :)

To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/1881dae6-4ce1-421c-8c4a-091518cbd7a3%40googlegroups.com.

Durand, Gustavo

unread,
Sep 10, 2019, 12:46:13 PM9/10/19
to dataverse...@googlegroups.com
I don't think the db idea would work, because the code creates the object with restricted = false and that would override the default anyway.

It's not the opposite of ":PublicInstall" because that requires all files to always be public.

This should just be a setting in the settings table ":RestrictFilesByDefault" (or something like that) with the default being false. It would be straightforward to check this code when you upload a dataset.

Do note that this will have to change once we add DataTags support, but I imagine we could have "defaultDataTag" and migrate the above from "false" to "blue" and "true" to "yellow".

The other thing to note is that currently when a user does restrict a file, we "nudge" the user to allow request access and / or add terms. We'd lose this in the case of default restricted, so another addition is that it may be worth considering nudging them at upload time, in this scenario.

Hope these thoughts help,
Gustavo

On Tue, Sep 10, 2019 at 10:41 AM Durbin, Philip <philip...@harvard.edu> wrote:
Ok. Somebody, please create an issue at https://github.com/IQSS/dataverse/issues :)

On Tue, Sep 10, 2019 at 9:54 AM Stefan Kasberger <kasberge...@gmail.com> wrote:
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/1881dae6-4ce1-421c-8c4a-091518cbd7a3%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/CABbxx8Gb3LY6LZc9FXF15Zz5Dz3C%3D2vj9HB0Swwm3W%3Dy3q6a-g%40mail.gmail.com.

MM - ADA

unread,
Sep 10, 2019, 7:31:05 PM9/10/19
to Dataverse Users Community


On Wednesday, September 11, 2019 at 2:46:13 AM UTC+10, Gustavo Durand wrote:
I don't think the db idea would work, because the code creates the object with restricted = false and that would override the default anyway.
   => Yes, I had forgotten this as I had tried in the very early stages of our installation to set the default for restricted = true and it kept getting overriden by the code.
 

It's not the opposite of ":PublicInstall" because that requires all files to always be public. 
    => Ok.
 
This should just be a setting in the settings table ":RestrictFilesByDefault" (or something like that) with the default being false. It would be straightforward to check this code when you upload a dataset.
=> Ok - sounds reasonable.

 
Do note that this will have to change once we add DataTags support, but I imagine we could have "defaultDataTag" and migrate the above from "false" to "blue" and "true" to "yellow".
=> If this is true, I am not sure it will be worth the effort to make the change. Or it might be - will have to look at it.
 

The other thing to note is that currently when a user does restrict a file, we "nudge" the user to allow request access and / or add terms. We'd lose this in the case of default restricted, so another addition is that it may be worth considering nudging them at upload time, in this scenario.
   => Allowing people to restrict on upload, and add the request access/terms, would be ideal without having to go into Edit-> Restrict every time. Click reduction. And error reduction (for ADA anyway).
 
Hope these thoughts help,
Gustavo

 
Thanks Gustavo! 

Sebastian Karcher

unread,
Sep 10, 2019, 9:14:06 PM9/10/19
to Dataverse Users Community
This would be useful for us, too. Thanks for the idea&discussion. I've created the issue on gh as requested: https://github.com/IQSS/dataverse/issues/6158

Tania Schlatter

unread,
Sep 11, 2019, 11:53:24 AM9/11/19
to dataverse...@googlegroups.com
Thanks for opening the issue, Sebastian. To share with Marina and others, Mercè and I had a poster at the Dataverse community meeting that put forth the idea of data being restricted, or yellow, by default unless changed by the depositor in a Dataverse installation with DataTags. When published, the datatag could remain yellow provided there is a DUA included, or could be changed to blue or green. A small version of the poster is attached. 

- Tania



--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/856b78df-3776-4a7e-9640-2a7aec1a849b%40googlegroups.com.
integrating_DataTags_2019_sm.pdf
Reply all
Reply to author
Forward
0 new messages