Archival feature for D. ataverse

[Iglezakis, Dorothea]

Dear Dataverse community,

we would like to offer the possibility to archive datasets (versioned, but unpublished) for a certain period of time (up to 10 years) in our dataverse instance (https://darus.uni-stuttgart.de). Archived datasets should be accessible like unpublished datasets (to all persons with ViewUnpublishedDataset permissions). We want to use this for clean up tasks to delete datasets that are neither published nor archived after a year. 

We started to look into the code and found a version state named ARCHIVED (f.e. https://github.com/IQSS/dataverse/blob/28a9d441eeb2e183bf9b0b083e89208a5e860b3c/src/main/java/edu/harvard/iq/dataverse/DatasetVersion.java#L596) but did not find any logic around this state or futher documentation (but perhaps we did not look properly).

Are there any plans to use this state? Or are there already any developments or implementation of an archival feature/workflow?

Does anyone have similar plans or requirements?

Thanks and best,

Doro (in the name of the DaRUS-Team)

__

Dr. Dorothea Iglezakis
FoKUS - Kompetenzzentrum für Forschungsdaten
IZUS/Universitätsbibliothek Stuttgart
Holzgartenstr. 16
70174 Stuttgart

Tel.: 0711/685-83648
Email: dorothea....@ub.uni-stuttgart.de

[Philip Durbin]

Hi Doro,

From a quick look at the code, I don't think that "ARCHIVED" state is used much. It looks like it was carried over from the last generation of Dataverse (DVN 3.x): https://github.com/IQSS/dvn/blob/3.6.2/DVN-root/DVN-web/src/main/java/edu/harvard/iq/dvn/core/study/StudyVersion.java#L45 . I'm not sure how to get a dataset into that state.

Generally speaking, we've been trying to list archive efforts here: https://guides.dataverse.org/en/6.2/admin/integrations.html#research-data-preservation

You'll find links to Archivematica and BagIt there. In the future, we'll be happy to list your new endeavor!

I hope this helps!

Phil

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/8aa1408665a84c54904b4a6918364b8f%40ub.uni-stuttgart.de.

--

Philip Durbin
Software Developer for http://dataverse.org
http://www.iq.harvard.edu/people/philip-durbin

[James Myers]

I don’t know of any support specifically for dealing with draft datasets. It sounds like the deaccessioned state might do ~what you want – it hides the data from the public but let’s the author see it, but it is only available after publication.

 

The archival bag work is less directly relevant. It makes a single zip file copy of the data and metadata that could be stored in a backup/archiving system, but it doesn’t remove the live dataset from Dataverse and doesn’t change its visibility, and it is setup to work for published versions. It’s possible to imagine archival bags being extended to be a way to store read-only datasets that could be viewed but would no longer be in the database (and would have to be re-imported to be editable again.) The basic idea there is to have code similar to what’s in the Zip Previewer that would show you metadata and allow download of individual files from within the zip without unpacking it. That would make them smaller, easier to put in colder storage, etc.

 

One cheap option – the Curation Status Labels functionality could have one label for “Archived” (or “Archived through 2034”, etc. if you want to track dates). It would enable your curators to see that the draft dataset is archived and get a report listing all the datasets in that state. Aside from the label though, the dataset would just remain a normal draft dataset.

 

--Jim

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/CABbxx8Gy5zaH%3D7UzfJZ9E%3Dx8F%3DoqS-FHivif3aETwQzPzBwe5w%40mail.gmail.com.

[Dorothea Iglezakis]

Dear Phil, dear Jim,

thanks a lot for your answers and input. Our first approach to get what we want is indeed to reuse/misuse/copy the deaccessioned status for the visibility and version behaviour and add additional information with an archival metadata block. We will keep you informed about the progress of our activities in this direction.

Best,

Doro

[Janet McDougall - Australian Data Archive]

hi All

I didn't think there was versioning available for unpublished datasets?  Unless this is a new feature?

Thanks

Janet

[Philip Durbin]

Sorry, versioning before a dataset is published is not supported. Please see this almost 10 year old issue: https://github.com/IQSS/dataverse/issues/1002

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/654cdba1-f96e-4e20-9016-f570332e6604n%40googlegroups.com.

[Asbjørn KU-IT]

Hello

This is a very interesting discussion and potentially a very useful addition to Dataverse for most repositories.

We are also interested in creating an "archival metadata block". This metadata block could not only be used for registering if data has been archived, but could also be used to register if the data "should" be archived, meaning an archival appraisal employee (archivist) should assess the data for archiving before any archiving. This is specifically something we are looking into implementing through a custom metadata block, but essentially if Dataverse had an archival metadata block "out-of-the-box" to be enabled, this would be a much better approach for us.

It would also be interesting to look at how this could work with workflows or API integration - since the receiving archive could be a National Archive or it could be an integration with archival software such as Archivematica inside your own organisation.

Please see this as interest in this feature, and please keep us in the loop of any further progress. We would also, potentially, be interested in contributing to the creation - or reviewing - of an archival metadata block.

Regards

Asbjørn, Uni Copenhagen IT Dept.

[Iglezakis, Dorothea]

Dear Asbjørn,

our archival metadata block is available on https://github.com/izus-fokus/darus-metadata-blocks/blob/master/archival.tsv So far, there is no implementation/workflow around this and the metadata block is not really used. So we are very open for any kind of collaboration on this metadata block  or the whole archiving feature idea.

Kind regards,

Doro (FoKUS Stuttgart)

---

Von: dataverse...@googlegroups.com <dataverse...@googlegroups.com> im Auftrag von Asbjørn KU-IT <a...@adm.ku.dk>
Gesendet: Mittwoch, 15. Mai 2024 13:36:09
An: Dataverse Users Community
Betreff: Re: [Dataverse-Users] Archival feature for D. ataverse

 

You received this message because you are subscribed to a topic in the Google Groups "Dataverse Users Community" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/dataverse-community/2wi1EnQwVOM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to dataverse-commu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/49525907-b49d-4d23-9d98-1407de53b4dan%40googlegroups.com.

[Asbjørn KU-IT]

Great work. I took at look at the block and made some changes in my fork: https://github.com/Asbjoedt/darus-metadata-blocks/blob/archival/archival.tsv

I added some fields, changes the wording of some fields and deleted the field "archiveActiveUntil". The deleted field seem to me to be the same "archiveArchivedFrom" just worded differently, or am I msitaken?

Pease give any comments, if possible.

@Philip Durbin, @Jim Myers. Are you interested in receiving a PR on an archival metadata block for Dataverse?

Regards Asbjørn

[Philip Durbin]

Sure, a pull request sounds nice. Rather than enabling this new block by default, we would very likely list it under "experimental metadata", like this: https://guides.dataverse.org/en/6.2/user/appendix.html#experimental-metadata

There you'll find a metadata block that Doro and others worked on for CodeMeta. Here's the PR: https://github.com/IQSS/dataverse/pull/7877

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/36c51d9b-ba36-46ce-81bb-27b5a6e5d5a2n%40googlegroups.com.