Propoed change to fcrepo server-managed triples

[Durbin, Michael (md5wz)]

One convenient feature of Fedora (pre-4.0 and the current community implementation) is automatic tracking of certain metadata including creation dates, modification dates and creating user and modifying user.

In fedora 2.x and 3.x these fields couldn't be directly set, but instead were maintained by the repository in response to repository operations. Furthermore there was a lossless and documented serialization format (and underlying storage format) in which these properties appeared: FOXML As such, when ingesting by posting FOXML those values would be set as part of an ingest. Therefore, when you ingested from FOXML, the recorded created date for datastreams and such were NOT the date you ingested the foxml, but the values from the FOXML.

In working on the export/import effort, there's a desire to achieve similar functionality. Our current construction of "fedora managed triples" prevents us from ever modifying these values as part of normal REST operations. In determining how best to support a lossless export/import process the least objectionable option from my point of view would be releasing some properties from being fedora-managed in the strictest sense. So instead of having fedora:created, fedora:lastModified, fedora:lastModifiedBy and fedora:createdBy be server-managed triples, instead they're (mostly) normal triples that can be manipulated like any other, except when not present in requests may be set automatically as a convenience feature of the fedora community implementation. Whether other implementations have this feature is up to them.

This recharacterization and relaxing of the constraints currently in the fedora community implementation is backwards compatible and arguably a non-breaking change.

While some might argue that they depend on those triples as an unimpeachable audit trail, I would point out that the fcrepo-audit effort provides a more comprehensive and reliable source of that information.

While a cursory examination suggests that relaxing the rules for just those triples is enough to allow an unversioned resource to be roundtripped (export followed by import) without apparent differences, the challenges posed by versioning require a more substantial change. After more thorough review of the current implementation and the nearly-completed spec, I'll try to make a recommendation for that next week.

In the mean time, I'd like to gauge the level of support or revulsion at this proprosed change to the community implementation (and possibly the spec if it enumerates fcrepo server managed triples).

If this change is as uncontroversial as I hope, it would be great to get it implemented in advance of the import/export sprint in May so that developers and testers have a version of fedora against which to implement the updated import/export tooling.

Thanks for your attention and feedback.

-Mike Durbin

[Adam Wead]

Mike,

Thanks for writing this up and bring it to everyone's attention.

So, I have a personal love-hate relationship with this particular issue. When Penn State migrated to Fedora 4, it was 4.0 and we had existing content coming from Fedora 3 with its original timestamps. We were faced with the problem of how to preserve the original timestamps in the new Fedora with its server-managed triples. This was the beginning of the hate part of the relationship. I wanted to override those dates, but couldn't.

We elected to use an additional predicate from Premis that identified the date of the original creating application, and we were also able to use this for the different versions of the resource as well (IIRC). This was for binary files, so it was only fcr:metadata nodes. In the end, this worked out pretty well for us. It had the additional benefit of using the auto-timestamped creation to identify the migration date, and in general seemed to work out for the best. Now we're in the love part of the relationship.

If I was reading this email three years ago when we were undertaking our migration, I would have responded to this thread with an enthusiastic thumbs up. Now, I'm a little hesitant. I think it's useful, maybe even required, to have these kinds of unassailable properties. However, your point about an identical-looking import/export process is extremely pertinent and wasn't really on forefront previously, so maybe that alone would tip things in that direction.

Having seen the issue from both sides, I'm more ambivalent now. However, if Fedora would happily allow a request to update an original creation or modification date, I would be wary every time I do anything, for I now have more proverbial rope with which to hang myself.

Just my thoughts!

Happy Friday,

...adam

> --
> You received this message because you are subscribed to the Google Groups "Fedora Tech" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to fedora-tech...@googlegroups.com.
> To post to this group, send email to fedor...@googlegroups.com.
> Visit this group at https://groups.google.com/group/fedora-tech.
> For more options, visit https://groups.google.com/d/optout.

[Durbin, Michael (md5wz)]

Thanks for the feedback Adam!

While I share some your concerns, the current motivation is that if there's no way to export content from one fedora instance and import it back into another instance without detectable differences between the resources then we have no way to effectively backup and restore content. That eliminates huge swaths of use cases for Fedora. (The current fcr:backup, fcr:restore is not part of the spec and will not work across implementations, and is relatively inflexible in that you can backup everything or nothing, you cannot export a lossless representation of a resource or subgraph)

Other approaches (like removing those properties from the RDF that represents a fedora resource) represents a huge breaking change and a significant loss of functionality.

As a clarification, if we decide to change the way the community implementation treats these triples we could take steps to address your concerns about having to be suspicious about the validity of these values if opened up for user updates. Some options that come to mind include:

- make this behavior optional, it would allow one to configure one's repository one way to allow for imports (restoration of backups made using the export utility or for migration) and another for normal use. This would truly make the feature a non-breaking change since a setting would ensure the current behavior.

- make this operation privileged where only administrative users can update these properties. This may not be sufficient for people who don't use WEBAC or give their front-end application administrative privileges (as we have to due to some webac compatibility issues with the hydra code we're using for some of our applications built on fedora)

- limit the ability to update these operations to PUT requests. PATCH updates to these properties smacks of mischief.

I'd support implementing all three of these constraints on the described behavior change, though admittedly I haven't evaluated the difficulty of implementing any of them.

-Mike
________________________________________
From: fedor...@googlegroups.com [fedor...@googlegroups.com] on behalf of Adam Wead [amste...@gmail.com]
Sent: Friday, April 28, 2017 3:14 PM
To: Fedora Tech
Subject: Re: [fedora-tech] Propoed change to fcrepo server-managed triples

[Joshua Westgard]

We are in favor of this change because it is necessary for migration and for lossless restore from serialized backups.  Previously, we also had concerns, similar to what Adam describes, about the loss of the resource creation date in the migration from an earlier Fedora to Fedora 4, but reached a similar conclusion that such a property is better captured in a different location than the server-managed fcrepo property.  And we will likely continue to use a separate property going forward -- even if changing the server-managed property becomes possible -- because the migration from earlier Fedoras to Fedora 4 represents such fundamental transition-point in the resource life-cycle.

We could work with any of the three options for placing constraints on this behavior that were mentioned by Mike.

Josh

[Esmé Cowles]

I am +1 on the change to allow modifying the system dates, and I agree that the three limits outlined by Mike seem reasonable to me. I particularly like the idea of allowing configuration to disable the behavior, which seems like a good way to accommodate anyone who wants to keep the current behavior.

-Esmé

[Adam Wead]

I'm +1 to all that's been said. It makes complete sense to override these properties if you need to have a matching export/import process. But, as Mike has enumerated, there need to be some safeguards. As long as I can't overwrite a creation date through "typical" repository operations, then I'm fine.

...adam

[Andrew Woods]

Hello Mike,

Thanks for raising the topic, considerations and options. I think there are strong arguments for support of lossless round-tripping of repository backups; and the non-controversial, backwards-compatible approach of making this functionality optional by configuration seems very reasonable.

Here is a ticket under which this work should proceed.

https://jira.duraspace.org/browse/FCREPO-2432

Thanks,

Andrew

>> To unsubscribe from this group and stop receiving emails from it, send an email to fedora-tech+unsubscribe@googlegroups.com.

>> To post to this group, send email to fedor...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/fedora-tech.
>> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups "Fedora Tech" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to fedora-tech+unsubscribe@googlegroups.com.

> To post to this group, send email to fedor...@googlegroups.com.
> Visit this group at https://groups.google.com/group/fedora-tech.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Fedora Tech" group.

To unsubscribe from this group and stop receiving emails from it, send an email to fedora-tech+unsubscribe@googlegroups.com.