Can we talk about UNF?

[sebastiank...@u.northwestern.edu]

This might be controversial, but I at least wanted to bring it up:

UNF is a really elegant concept and I appreciate why it was included in the original King & Altman data citation article and then into the Dataverse recommended citation. I think, however, that it should be retired as part of the recommended citation for three reasons:

1. Standardization: UNF is only used by Dataverse repositories. As far as I know, no other repository uses it and UNF (or other hashes) are not included in recommended citations as described by Datacite or the RDA data citation working group

2. Limited use: UNF applies principally to numeric, tabular data. An increasing amount of data are not that -- this is obviously true for QDR-type qualitative data, but also applies to JSON, GIS-data, all sorts of imaging formats such as MRI data, etc.

3. False security: Since UNF only applies to tabular files, it gives a false sense of security: a dataset could, e.g., be missing an entire (non-tabular) file or have a scale description reversed in the codebook and still show up with the same UNF

We can (and likely will) discontinue use of UNF at QDR, where 2.) ways more heavily than for some other repositories, but I figured it might be useful to start this post to see if there would be any inclination to re-consider this more generally.

Looking forward to see what people think,

Best,

Sebastian

[leo...@g.harvard.edu]

I don't know if it's truly a "controversial" idea, but I entirely understand the sentiment. 

It really was an elegant concept... but in some ways it got stuck in that "concept" stage. (The project has not been actively maintained since Dr. Altman left Harvard). That by itself makes me unsure about continuing to use and rely on it. 

I don't have a strong opinion on whether it's "false security" - but I definitely understand your argument (item 3.). (It's entirely fair to point out that UNF was developed and added to D̶a̶t̶a̶v̶e̶r̶s̶e̶ the grandmother of Dataverse back when it was dedicated solely to Social Sciences and when tabular data could be considered the primary payload of any dataset; which is no longer true). But I have to acknowledge that it's not entirely clear just how much security it adds even when it comes to tabular data. Having a secure, semantically aware signature of your data is a great idea. But the value of it is not clear when there is no easy way to recalculate and verify it outside of Dataverse. (item 1. on your list). 

To clarify, I'm not actively arguing in favor of retiring it myself. And from what you are saying in your post, you are not necessarily requesting its complete retirement either - but maybe an installation-wide configuration option "donotuseUNF=true"? 

Best,

-Leonid

[Crosas, Mercè]

Sebastian,

I agree with your points. Maybe a configuration-option, as Leonid suggested, is a good idea.

It would be good to hear from other Dataverse installations from the community. Preferences? Concerns? 

We can't change citations from datasets versions already published, but it could be a change for future citations.

Best,

Mercè

Merce

--

Mercè Crosas, Ph.D.

University Research Data Management Officer, Harvard University Information Technology

Chief Data Science and Technology Officer, Institute for Quantitative Social Science

Harvard University

mcr...@g.harvard.edu | @mercecrosas | scholar.harvard.edu/mercecrosas

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/8dc16ff7-3420-429c-a5f3-e09dfb7e47b2n%40googlegroups.com.

[Sebastian Karcher]

Thanks Leonid,  Mercè,

For engaging on this. I'm glad it isn't as controversial as I thought it might be!

I'd be perfectly happy with a config option -- perhaps one to default to off to reflect the lack of usage beyond existing DV installations?

I'm curious why you think existing citations can't be changed, Mercè? I was leaning towards removing this from existing QDR citations, but will not if there's a strong reason to keep it. My thinking is that it's a recommended citation in the first place, and there's no reason that shouldn't change (just like we've changed RIS content in the past, e.g.). The actual item metadata and its UNF wouldn't change, so people could still verify the UNF from existing citations.

Also looking forward to other folks' comments.

All the best,

Sebastian

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/CA%2B_j1Jm-JmDQhUFH3AuZsHWviZ8J2Q1%2Bn_2D_-vyS0n8VSnnxA%40mail.gmail.com.

--

Sebastian Karcher, PhD
www.sebastiankarcher.com

[Philip Durbin]

Technically, there is one non-Dataverse implementation of UNF that I'm aware of, and it's implemented in R rather than Java. However, like the Java-based UNF implementation used in Dataverse, it hasn't been touched in years: https://github.com/leeper/UNF

If UNF had truly taken off, we probably would have seen even more implementations in various languages by now.

Hope this helps,

Phil

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/CAOSYSD7VECcZo9Ss%3DWD13GSAcTwN7VYND3tdjgnO7Ht85PuKrw%40mail.gmail.com.

--

Philip Durbin
Software Developer for http://dataverse.org
http://www.iq.harvard.edu/people/philip-durbin

[Crosas, Mercè]

Hi Sebastian,

It might not be critical to keeping the UNF from old citations, but my reasoning was that if a citation has been published and potentially used already in publications or other references, it's not a good practice to change it unless something has changed in the dataset. You are right that it can still be verified using the metadata that you are referencing the same dataset. It's nevertheless a bit confusing by data users to see that the citation itself has changed for the same object, so in general, I would recommend to keep the old ones and change it moving forward.

 I'm all open to hearing other arguments and views from the community on this.

Merce

--

Mercè Crosas, Ph.D.

University Research Data Management Officer, Harvard University Information Technology

Chief Data Science and Technology Officer, Institute for Quantitative Social Science

Harvard University

mcr...@g.harvard.edu | @mercecrosas | scholar.harvard.edu/mercecrosas

On Tue, Oct 6, 2020 at 11:03 PM Sebastian Karcher <kar...@u.northwestern.edu> wrote:

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/CAOSYSD7VECcZo9Ss%3DWD13GSAcTwN7VYND3tdjgnO7Ht85PuKrw%40mail.gmail.com.

[sebastiank...@u.northwestern.edu]

Thanks everyone -- there seemed to be enough emerging consensus to open a ticket:

https://github.com/IQSS/dataverse/issues/7328

[danny...@g.harvard.edu]

Thanks Sebastian for creating the ticket. We'll probably have some additional thoughts and there's a few other people that we may want to tag in for the discussion. The ticket is a good spot to coordinate. 

- Danny

[Donald Sizemore II]

Hello dataverse-community,

At the risk of reviving an old thread: the consortium has a member institution who would like to implement Leonid's proposed configuration option to disable the use of UNFs (or, minimally, to disable the display of UNFs).
How many installations would be interested in such a switch, and perhaps pooling funding to pay for the implementation? (feel free to reply to me directly for the comfort of everyone else on the list)

Thank you,

Don

[Sherry Lake]

Yes.... would love to see UNF go away.

How many votes do I get?

Is there any background on why it is there in the first place. I know when it appears (in datasets with ingested - "tabular" files). But that's all I know.

--

Sherry Lake

UVA Dataverse http://dataverse.lib.virginia.edu

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/a4668c85-b9a1-481d-b52d-8dd647d3d467n%40googlegroups.com.

[Sebastian Karcher]

As the qualitative vanguard ;) , we've removed UNF shortly after my above post, so I'd obviously be very supportive in defaulting the pref to not using it across DV installations and I'd be happy for Jim to share the patch he used in our case.

The reason it was used originally is that back in ~2006 DV was a place for quantitative social science datasets; data citations were virtually unheard of, and Micah and Gary proposed a standard for data citations that included the very elegant file-format agnostic UNF that Micah and co-authors had developed back in 2003. The rest is path dependency/institution stasis, which are completely understandable, but it's really time to retire its use at least in recommended citations.

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/CADL9p-VLgOC5zsT3PVPtGP%3Ddgwu%2B3Adw7_CDUFEaonZVGj8erA%40mail.gmail.com.

[Don Sizemore]

Sebastian,

Jim's patch removed portions of code: https://github.com/QualitativeDataRepository/dataverse/issues/64#issuecomment-902243204

but we're hoping to introduce a setting to let installations decide.

Thank you,

Don

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/CAOSYSD7PwN8H%2BZfV1KdXPL%2B2vNU-L5i5gr%3DHO37Yo4DBH6VpHg%40mail.gmail.com.