blind peer review of dataset

[Meghan Goodchild]

Hello there,

We had a question from a librarian who would like to support a researcher with a dataset for blind peer review using the private URL function. They are requesting to create a dummy user account so that the depositor is anonymous. We have an institutional Dataverse set up, so we need to give permission to users to deposit in the root so that there is no institutional affiliation associated with the dataset for the blind review. We're concerned about facilitating dummy user accounts and ensuring that these datasets aren't left abandoned.

Does anyone have experience with this? Any best practices or suggestions about setup and workflow would be appreciated! I've found a github ticket related to this question and it would be great to have a system managed way to hide the metadata and depositor information during blind peer review: https://github.com/IQSS/dataverse/issues/1724

Thanks,

Meghan 

Scholars Portal Dataverse

[Sebastian Karcher]

I would be concerned about dummy accounts, too. Do you need those for your purposes, though? You can just edit the author information in the dataset that you're sending for review. Someone with the review URL wouldn't be able to see which account created the dataset, I believe.

[Meghan Goodchild]

Hi Sebastian,

Although you can add anonymous metadata information about the dataset, the depositor name (from the account that creates the dataset) is still visible under the versions tab and the email address of the contact account (required) can be exported via the metadata. So, this does not meet the needs of the researcher/librarian and we are concerned about managing a dummy account that would withhold this information.

I think we would be interested in a way for the system to suppress account information during the blind review until it is ready to be published or deleted.

Thanks,

Meghan

[Philipp at UiT]

Hi!

We have also been discussing this issue. Our current solution - though not tested out - is that a curator would create the dataset on behalf of the researcher, so that no information about the researcher will be visible, not event in the versions tab. However, this is quite cumbersome, and we'd really appreciate anonymous versions of unpublished datasets as an integrated feature of Dataverse.

Best, Philipp (DataverseNO)

[Durand, Gustavo]

Hi,

(I thought I hit send on this yesterday, only to see it is still now in a draft stat - sorry about that!)

So I unfortunately can't think of a great workaround right now for the depositor on the versions tab. Would you mind creating a new issue for this? It would be straightforward to add code that hides the contributor info on the version tab but we'd still have issues with datasets that are using Private URLs for version 2* and beyond. (the user could just go to an earlier version that is published and we wouldn't be hiding info there)

* are private URLs used for v2 of datasets or is this an edge case we don't really need to worry about?

Thanks,

Gustavo

On Mon, Jun 3, 2019 at 12:47 PM Meghan Goodchild <meghan.good...@gmail.com> wrote:

Hi Sebastian,

Although you can add anonymous metadata information about the dataset, the depositor name (from the account that creates the dataset) is still visible under the versions tab and the email address of the contact account (required) can be exported via the metadata. So, this does not meet the needs of the researcher/librarian and we are concerned about managing a dummy account that would withhold this information.

I think we would be interested in a way for the system to suppress account information during the blind review until it is ready to be published or deleted.

Thanks,

Meghan

On Monday, June 3, 2019 at 11:22:27 AM UTC-4, Sebastian Karcher wrote:

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/a702fa4d-45b1-4068-9f9d-669c7d0db1f0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Durand, Gustavo]

And since I wrote this, Phillips idea came in - yes the curator creating the dataset is a currently available, yet cumbersome workaround. Having Dataverse truly anonymize the dataset is clearly better, which I think we can accomplish by hiding contributor info in the versions tab (or maybe the entire tab altogether?) and other similar tweaks with export / persistent id, etc.

Once the issue is created, we can determine the full scope to handle this.

On Tue, Jun 4, 2019 at 11:09 AM Durand, Gustavo P <gdu...@iq.harvard.edu> wrote:

Hi,

(I thought I hit send on this yesterday, only to see it is still now in a draft stat - sorry about that!)

So I unfortunately can't think of a great workaround right now for the depositor on the versions tab. Would you mind creating a new issue for this? It would be straightforward to add code that hides the contributor info on the version tab but we'd still have issues with datasets that are using Private URLs for version 2* and beyond. (the user could just go to an earlier version that is published and we wouldn't be hiding info there)

* are private URLs used for v2 of datasets or is this an edge case we don't really need to worry about?

Thanks,

Gustavo

On Mon, Jun 3, 2019 at 12:47 PM Meghan Goodchild <meghan.good...@gmail.com> wrote:

Hi Sebastian,

Although you can add anonymous metadata information about the dataset, the depositor name (from the account that creates the dataset) is still visible under the versions tab and the email address of the contact account (required) can be exported via the metadata. So, this does not meet the needs of the researcher/librarian and we are concerned about managing a dummy account that would withhold this information.

I think we would be interested in a way for the system to suppress account information during the blind review until it is ready to be published or deleted.

Thanks,

Meghan

On Monday, June 3, 2019 at 11:22:27 AM UTC-4, Sebastian Karcher wrote:

--

You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/a702fa4d-45b1-4068-9f9d-669c7d0db1f0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/CAF2sSefhK-kSeYstKC5HHmnobqYcYtfS8%3Df1AXo6Hjq7ja%2Bo%3Dg%40mail.gmail.com.

[Philipp at UiT]

* are private URLs used for v2 of datasets or is this an edge case we don't really need to worry about?

As far as I remember private URLs can only be used with unpublished versions.

Best, Philipp

To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.

To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/a702fa4d-45b1-4068-9f9d-669c7d0db1f0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.

To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.

[Philip Durbin]

Private URLs are not limited to datasets that have never been published. The following scenario should work fine:

- User publishes version 1.0.

- Time passes.

- User creates a draft that will be some day be published as version 1.1 or 2.0.

- User creates a Private URL.

Here's a test that asserts this: https://github.com/IQSS/dataverse/blob/v4.14/src/test/java/edu/harvard/iq/dataverse/api/DatasetsIT.java#L969

I hope this helps,

Phil

p.s. Please keep up the chatter until there's rough consensus on the approach that should be taken for blind peer review in terms of making it a first class feature. Perhaps it would be easier to implement if we say that (unlike Private URLs) the (future) blind peer review feature only works with datasets that have never been published. Sometimes we call these "pre-1.0 drafts".

To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.

To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/a702fa4d-45b1-4068-9f9d-669c7d0db1f0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.

To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.

To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/CAF2sSefhK-kSeYstKC5HHmnobqYcYtfS8%3Df1AXo6Hjq7ja%2Bo%3Dg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.

To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.

To post to this group, send email to dataverse...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/8e392a61-73c4-49d2-9115-51b8f0e71b3e%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Philip Durbin
Software Developer for http://dataverse.org
http://www.iq.harvard.edu/people/philip-durbin

[Philipp at UiT]

Thanks for clarifying this, Phil. I have never tried to create a private URL to a draft version of a published dataset.

Perhaps it would be easier to implement if we say that (unlike Private URLs) the (future) blind peer review feature only works with datasets that have never been published. Sometimes we call these "pre-1.0 drafts".

I agree. I think for single blind peer review (only the reviewer(s) are anonymous), the private URL feature is probably sufficient. But for double blind peer review, I think we need a peer review feature.

The Open Access publishing services at my uni, Septentrio (https://septentrio.uit.no) have just decided to use our Dataverse instance, DataverseNO, (https://dataverse.no) as their repository for underlying data to their publications. Currently, the journals included in Septentrio use double blind peer review. Until a peer review functionality in Dataverse is in place, we have decided that datasets to be peer reviewed in Septentrio journals should be created by a curator, so that no information about the author(s) will be visible in the dataset.

Maybe, we can discuss this further later today at the Dataverse Community pre-meeting :)

Best, Philipp

To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.

To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/a702fa4d-45b1-4068-9f9d-669c7d0db1f0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.

To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.

To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/CAF2sSefhK-kSeYstKC5HHmnobqYcYtfS8%3Df1AXo6Hjq7ja%2Bo%3Dg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.

To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.

To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/8e392a61-73c4-49d2-9115-51b8f0e71b3e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Crosas, Mercè]

That's a good approach, Philipp. Thanks for the information.

To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.

To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/a702fa4d-45b1-4068-9f9d-669c7d0db1f0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.

To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.

To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/CAF2sSefhK-kSeYstKC5HHmnobqYcYtfS8%3Df1AXo6Hjq7ja%2Bo%3Dg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.

To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.

To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/8e392a61-73c4-49d2-9115-51b8f0e71b3e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Philip Durbin
Software Developer for http://dataverse.org
http://www.iq.harvard.edu/people/philip-durbin

--

You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.

To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.

To post to this group, send email to dataverse...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/6dc223c8-7f6d-4dad-9efe-1bae1bf59ba4%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Mercè Crosas, Ph.D.

Harvard University's Research Data Officer, Office of Vice Provost for Research

Chief Data Science and Technology Officer, Institute for Quantitative Social Science

mcr...@g.harvard.edu | @mercecrosas | scholar.harvard.edu/mercecrosas