Memento + LDP and mutation
63 views
Skip to first unread message
soro...@gmail.com
Sep 14, 2016, 3:47:07 PM9/14/16
to memen...@googlegroups.com
Hi, Memento crew!
I'm writing from Fedora Commons [1], a digital object repository framework in wide use globally at institutions-of-memory. Our latest version [2] is centered on the Linked Data Platform specification [3]. We are offering the kinds of capabilities (fixity, audit, etc.) that our users desire in that context. One of those capabilities is versioning, and we are committed to using Memento to do that. But Memento only defines means by which to explore and retrieve versions, not create them, so we've got to extend a little somehow. To that end, our central ideas for versioning are as follows:
1) A given Fedora resource R will have associated with it by the usual Memento-defined means another resource that is both a TimeGate and a TimeMap for R. This is a pattern that is already described by Memento. We think that simplifying assumption is appropriate for us because in the context of a repository framework, it's unlikely that agencies other that that which is operating the repository will be taking much responsibility for offering Mementos. Not impossible, but unlikely.
2) That TimeGate/TimeMap will be simultaneously an LDP Container [4] (we are calling them LDP-Cv, "v" for "versioning") which contains (in the LDP sense) the Mementos of R. That means that we can use the LDP-specified modes of interaction to allow users to add contained resources to it, which amounts to adding versions.
So we extend Memento "by" LDP so that a TimeMap can be used both to discover and to create versions.
Does this sound reasonable to y'all? Are there pitfalls that we're missing? Is a TimeMap the right element to address for version creation? Another option might be the TimeGate, which for us is the same resource, but we want to develop a good general design. The TimeMap seemed more appropriate because it functions more like a container from the GET side of things. Other projects are designing for the interaction of Memento and LDP (including Apache Marmotta [5]) and we want to get this right. Ideally, we'd like to bring back the beginnings of a proposed extension to Memento for this kind of use.
---
A. Soroka
The University of Virginia Library
[1] http://fedorarepository.org/
[2] https://wiki.duraspace.org/display/FF/Fedora+Repository+Home
[3] https://www.w3.org/TR/ldp/
[4] https://www.w3.org/TR/ldp/#ldpc
[5] https://marmotta.apache.org/platform/versioning-module.html
I'm writing from Fedora Commons [1], a digital object repository framework in wide use globally at institutions-of-memory. Our latest version [2] is centered on the Linked Data Platform specification [3]. We are offering the kinds of capabilities (fixity, audit, etc.) that our users desire in that context. One of those capabilities is versioning, and we are committed to using Memento to do that. But Memento only defines means by which to explore and retrieve versions, not create them, so we've got to extend a little somehow. To that end, our central ideas for versioning are as follows:
1) A given Fedora resource R will have associated with it by the usual Memento-defined means another resource that is both a TimeGate and a TimeMap for R. This is a pattern that is already described by Memento. We think that simplifying assumption is appropriate for us because in the context of a repository framework, it's unlikely that agencies other that that which is operating the repository will be taking much responsibility for offering Mementos. Not impossible, but unlikely.
2) That TimeGate/TimeMap will be simultaneously an LDP Container [4] (we are calling them LDP-Cv, "v" for "versioning") which contains (in the LDP sense) the Mementos of R. That means that we can use the LDP-specified modes of interaction to allow users to add contained resources to it, which amounts to adding versions.
So we extend Memento "by" LDP so that a TimeMap can be used both to discover and to create versions.
Does this sound reasonable to y'all? Are there pitfalls that we're missing? Is a TimeMap the right element to address for version creation? Another option might be the TimeGate, which for us is the same resource, but we want to develop a good general design. The TimeMap seemed more appropriate because it functions more like a container from the GET side of things. Other projects are designing for the interaction of Memento and LDP (including Apache Marmotta [5]) and we want to get this right. Ideally, we'd like to bring back the beginnings of a proposed extension to Memento for this kind of use.
---
A. Soroka
The University of Virginia Library
[1] http://fedorarepository.org/
[2] https://wiki.duraspace.org/display/FF/Fedora+Repository+Home
[3] https://www.w3.org/TR/ldp/
[4] https://www.w3.org/TR/ldp/#ldpc
[5] https://marmotta.apache.org/platform/versioning-module.html
Herbert Van de Sompel
Sep 15, 2016, 3:29:39 PM9/15/16
to memen...@googlegroups.com, Herbert Van de Sompel
Hi Adam,
(A rather long mail. But this is an important topic!)
Thanks a lot for your mail. We are quite aware of Fedora Commons. We are thrilled that you have decided to support the Memento protocol with regard to versioning and very much appreciate that you want to get it right. You are correct that the protocol is about accessing resource versions and not about creating them. Yet I would not characterize what you plan to do as extending the Memento protocol but rather as:
- Adopting the protocol for accessing resource versions in an LDP environment. and
- Defining an approach for creating resource versions in that context.
(1) Regarding the idea of merging of the TimeGate (URI-G) and TimeMap (URI-T) associated with an Original Resource (URI-R):
We had come across this idea before in a Fedora API document and have discussed it at various points with Memento experts including Michael Nelson, Robert Sanderson (also an LDP aficionado), Martin Klein, members of my team, myself. The outcome of each conversation has been that this can't really be done:
=> A TimeGate and a TimeMap are conceptually very different resources:
- A TimeGate supports datetime negotiation as a means to access a temporal versions of an Original Resource. A TimeMap provides a version history.
- Interaction with a TimeGate leads to a representation of a prior version of an Original Resource; that representation can be of any media type. Interaction with a TimeMap leads to an overview of prior versions of an Original Resource; as per the Memento RFC [1], that overview is of MIME type application/link+format and more recently a JSON TimeMap format has been defined [2]. Most importantly, that overview is not a prior representation of the Original Resource, which is what one would expect when interacting with a TimeGate.
- Note that the W3C Data on the Web Best Practices [3] clearly maintains the distinction between TimeGate and TimeMap
=> Merging TimeGate and TimeMap would lead to technical confusion for Memento clients. At this stage I would like to keep explaining this aspect simple, basically indicating that interaction with a conflated TimeGate/TimeMap would necessarily need to simultaneously exhibit the behavior of a TimeGate and a TimeMap. This can lead to confusing/conflicting HTTP response headers and unintended responses:
- Think, for example, how a TimeMap would be retrieved from the TimeGate/TimeMap combo. Maybe the idea was to access it without an Accept-Datetime header? The RFC is clear that in this case, the TimeGate must provide the most recent Memento for the Original Resource. Maybe the idea was to access it without an Accept-Datetime header and with an Accept header that expresses "application/link-format". As per the RFC, the TimeGate should still respond with the recent Memento, maybe feeling sorry that it can not honor the content negotiation request for "application/link-format" but honoring the requirement in the RFC that datetime negotiation takes precedence over content negotiation.
- HTTP response headers would be truly confusing to clients in cases where the TimeMap itself would also be versioned and have its own associated TimeGate, say URI-G(T). If needed, details can be provided but maybe the other arguments are already sufficient. Maybe you do not intend to version TimeMaps in Fedora but who knows what other LDP platforms plan for the future. As you indicated correctly, we need to get it right.
(2) Regarding modeling a TimeGate/TimeMap as an LDP Container
=> Looking at this idea, we are rather convinced that it would be a good idea to model the LDP Container as a TimeMap (distinct from the TimeGate) to which one can POST:
- A GET on an LDP Container yields a list of “contained” resources. That actually is what a TimeMap does too. So, a TimeMap can be regarded a special LDP Container associated with an Original Resource that lists versions of an Original Resource.
- The Memento protocol is only about HEAD/GET and leaves POST etc undefined. So, clearly, the LDP approach to add a resource to an LDP container using POST is not in conflict with the Memento protocol when applied to an LPD Container that is a TimeMap. The only LDP “specialization” is that the resource that will be POSTed to the LDP Container is a version of the Original Resource. All good.
- The LDP spec includes the use of the Prefer request header for interaction with an LDP Container. This might be of interest to define selective access to version information from the TimeMap, e.g. for paging or to request versions only between specified start/end dates. This would actually be an extension of the Memento protocol that could also be supported by platforms other than LDP, e.g. web archives. Interesting!
=> The question remains what to do with the TimeGate. As per the above, it should be distinct from the TimeMap. There are two options:
- Original Resource <> TimeGate <> TimeMap - This is Pattern 2 of the RFC [4]. It is a typical implementation of the protocol when dealing with web archives, in that the Original Resource and the TimeGate live on different systems.
- Original Resource == TimeGate <> TimeMap - This is Pattern 1 of the RFC [5], whereby the Original Resource acts as its own TimeGate. It is an approach that can be used by platforms that contain their own resource version history, such as wikis, software versioning systems, etc. This might be of interest if you want to economize on the number of URIs involved. Of the 3 approaches listed for this Pattern in the RFC, it seems that Patterns 1.1 and 1.2 would apply to an LDP environment that deals with versions.
(3) Regarding Marmotta
We are aware of their Memento implementation and are currently having a close look regarding compliance with the protocol. We plan to post some issues to their repository in the days to come. It would be really nice if all LDP platforms would implement Memento and the addition of a new version in the same way. That would require a spec, of course. We would be interested to collaborate if such an effort would be launched.
I hope this helps. Please share reactions. We are most willing to further discuss, help, etc.
Thanks again for getting in touch!
Greetings
Herbert
[1] http://mementoweb.org/guide/rfc/
[2] http://mementoweb.org/guide/timemap-json/
[3] https://www.w3.org/TR/dwbp/
[4] http://mementoweb.org/guide/rfc/#Pattern2
[5] http://mementoweb.org/guide/rfc/#Pattern1
(A rather long mail. But this is an important topic!)
Thanks a lot for your mail. We are quite aware of Fedora Commons. We are thrilled that you have decided to support the Memento protocol with regard to versioning and very much appreciate that you want to get it right. You are correct that the protocol is about accessing resource versions and not about creating them. Yet I would not characterize what you plan to do as extending the Memento protocol but rather as:
- Adopting the protocol for accessing resource versions in an LDP environment. and
- Defining an approach for creating resource versions in that context.
(1) Regarding the idea of merging of the TimeGate (URI-G) and TimeMap (URI-T) associated with an Original Resource (URI-R):
We had come across this idea before in a Fedora API document and have discussed it at various points with Memento experts including Michael Nelson, Robert Sanderson (also an LDP aficionado), Martin Klein, members of my team, myself. The outcome of each conversation has been that this can't really be done:
=> A TimeGate and a TimeMap are conceptually very different resources:
- A TimeGate supports datetime negotiation as a means to access a temporal versions of an Original Resource. A TimeMap provides a version history.
- Interaction with a TimeGate leads to a representation of a prior version of an Original Resource; that representation can be of any media type. Interaction with a TimeMap leads to an overview of prior versions of an Original Resource; as per the Memento RFC [1], that overview is of MIME type application/link+format and more recently a JSON TimeMap format has been defined [2]. Most importantly, that overview is not a prior representation of the Original Resource, which is what one would expect when interacting with a TimeGate.
- Note that the W3C Data on the Web Best Practices [3] clearly maintains the distinction between TimeGate and TimeMap
=> Merging TimeGate and TimeMap would lead to technical confusion for Memento clients. At this stage I would like to keep explaining this aspect simple, basically indicating that interaction with a conflated TimeGate/TimeMap would necessarily need to simultaneously exhibit the behavior of a TimeGate and a TimeMap. This can lead to confusing/conflicting HTTP response headers and unintended responses:
- Think, for example, how a TimeMap would be retrieved from the TimeGate/TimeMap combo. Maybe the idea was to access it without an Accept-Datetime header? The RFC is clear that in this case, the TimeGate must provide the most recent Memento for the Original Resource. Maybe the idea was to access it without an Accept-Datetime header and with an Accept header that expresses "application/link-format". As per the RFC, the TimeGate should still respond with the recent Memento, maybe feeling sorry that it can not honor the content negotiation request for "application/link-format" but honoring the requirement in the RFC that datetime negotiation takes precedence over content negotiation.
- HTTP response headers would be truly confusing to clients in cases where the TimeMap itself would also be versioned and have its own associated TimeGate, say URI-G(T). If needed, details can be provided but maybe the other arguments are already sufficient. Maybe you do not intend to version TimeMaps in Fedora but who knows what other LDP platforms plan for the future. As you indicated correctly, we need to get it right.
(2) Regarding modeling a TimeGate/TimeMap as an LDP Container
=> Looking at this idea, we are rather convinced that it would be a good idea to model the LDP Container as a TimeMap (distinct from the TimeGate) to which one can POST:
- A GET on an LDP Container yields a list of “contained” resources. That actually is what a TimeMap does too. So, a TimeMap can be regarded a special LDP Container associated with an Original Resource that lists versions of an Original Resource.
- The Memento protocol is only about HEAD/GET and leaves POST etc undefined. So, clearly, the LDP approach to add a resource to an LDP container using POST is not in conflict with the Memento protocol when applied to an LPD Container that is a TimeMap. The only LDP “specialization” is that the resource that will be POSTed to the LDP Container is a version of the Original Resource. All good.
- The LDP spec includes the use of the Prefer request header for interaction with an LDP Container. This might be of interest to define selective access to version information from the TimeMap, e.g. for paging or to request versions only between specified start/end dates. This would actually be an extension of the Memento protocol that could also be supported by platforms other than LDP, e.g. web archives. Interesting!
=> The question remains what to do with the TimeGate. As per the above, it should be distinct from the TimeMap. There are two options:
- Original Resource <> TimeGate <> TimeMap - This is Pattern 2 of the RFC [4]. It is a typical implementation of the protocol when dealing with web archives, in that the Original Resource and the TimeGate live on different systems.
- Original Resource == TimeGate <> TimeMap - This is Pattern 1 of the RFC [5], whereby the Original Resource acts as its own TimeGate. It is an approach that can be used by platforms that contain their own resource version history, such as wikis, software versioning systems, etc. This might be of interest if you want to economize on the number of URIs involved. Of the 3 approaches listed for this Pattern in the RFC, it seems that Patterns 1.1 and 1.2 would apply to an LDP environment that deals with versions.
(3) Regarding Marmotta
We are aware of their Memento implementation and are currently having a close look regarding compliance with the protocol. We plan to post some issues to their repository in the days to come. It would be really nice if all LDP platforms would implement Memento and the addition of a new version in the same way. That would require a spec, of course. We would be interested to collaborate if such an effort would be launched.
I hope this helps. Please share reactions. We are most willing to further discuss, help, etc.
Thanks again for getting in touch!
Greetings
Herbert
[1] http://mementoweb.org/guide/rfc/
[2] http://mementoweb.org/guide/timemap-json/
[3] https://www.w3.org/TR/dwbp/
[4] http://mementoweb.org/guide/rfc/#Pattern2
[5] http://mementoweb.org/guide/rfc/#Pattern1
--
---
You received this message because you are subscribed to the Google Groups "Memento Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to memento-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Herbert Van de Sompel
Digital Library Research & Prototyping
Los Alamos National Laboratory, Research Library
http://public.lanl.gov/herbertv/
http://orcid.org/0000-0002-0715-6126
==
Digital Library Research & Prototyping
Los Alamos National Laboratory, Research Library
http://public.lanl.gov/herbertv/
http://orcid.org/0000-0002-0715-6126
==
soro...@gmail.com
Sep 23, 2016, 11:36:25 AM9/23/16
to memen...@googlegroups.com
I'm sorry to be replying to this so late-- it got shunted around in my email filters!
1) I can be brief here: I accept all of your arguments, and I'm confident that other Fedora folks also will. I do think that Fedora has got some additional assumptions available that mitigate some of the problems you point to, but as you say and we agree: "who knows what other LDP platforms plan for the future. ... we need to get it right." We'll get back to the drawing board and make sure we have a clear TG/TM separation.
2) Great! Sounds like our TimeMap/LDP-C idea isn't crazy, and that's by far the most useful idea we've had. I think we'll be most interested in Pattern 1 of the RFC, both because we are a repository framework and we can make some assumptions about the repository "owning" the lifecycle of resources within it and because our clients do have some limits on their appetite for URIs. {grin} We've had a lot of success populating Prefer to get useful behavior, and I don't think it would be hard to work up some examples of TimeMap paging, slicing, etc. for discussion. Paging is a little different than our extant examples, because we deal largely in RDF, for which ordering is fraught at best. But there's an inherent sensible ordering in Memento that we can use.
3) We are extremely interested in taking this forward in a way that everyone can use. (Again, sorry about the delay in response!) We're participating strongly in the LDPNext Community Group. I'm not sure whether that's a perfect venue for communicating with LDP implementors, but it might help avoid the combinatorial problem of getting everyone talking to everyone else.
---
A. Soroka
The University of Virginia Library
> To unsubscribe from this group and stop receiving emails from it, send an email to memento-dev...@googlegroups.com.
1) I can be brief here: I accept all of your arguments, and I'm confident that other Fedora folks also will. I do think that Fedora has got some additional assumptions available that mitigate some of the problems you point to, but as you say and we agree: "who knows what other LDP platforms plan for the future. ... we need to get it right." We'll get back to the drawing board and make sure we have a clear TG/TM separation.
2) Great! Sounds like our TimeMap/LDP-C idea isn't crazy, and that's by far the most useful idea we've had. I think we'll be most interested in Pattern 1 of the RFC, both because we are a repository framework and we can make some assumptions about the repository "owning" the lifecycle of resources within it and because our clients do have some limits on their appetite for URIs. {grin} We've had a lot of success populating Prefer to get useful behavior, and I don't think it would be hard to work up some examples of TimeMap paging, slicing, etc. for discussion. Paging is a little different than our extant examples, because we deal largely in RDF, for which ordering is fraught at best. But there's an inherent sensible ordering in Memento that we can use.
3) We are extremely interested in taking this forward in a way that everyone can use. (Again, sorry about the delay in response!) We're participating strongly in the LDPNext Community Group. I'm not sure whether that's a perfect venue for communicating with LDP implementors, but it might help avoid the combinatorial problem of getting everyone talking to everyone else.
---
A. Soroka
The University of Virginia Library
> For more options, visit https://groups.google.com/d/optout.
>
>
>
> --
> Herbert Van de Sompel
> Digital Library Research & Prototyping
> Los Alamos National Laboratory, Research Library
> http://public.lanl.gov/herbertv/
> http://orcid.org/0000-0002-0715-6126
>
> ==
>
>
>
>
> --
> Herbert Van de Sompel
> Digital Library Research & Prototyping
> Los Alamos National Laboratory, Research Library
> http://public.lanl.gov/herbertv/
> http://orcid.org/0000-0002-0715-6126
>
> ==
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups "Memento Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to memento-dev...@googlegroups.com.
>
> ---
> You received this message because you are subscribed to the Google Groups "Memento Development" group.
Herbert Van de Sompel
Sep 23, 2016, 11:40:18 AM9/23/16
to memen...@googlegroups.com, Herbert Van de Sompel
Thanks a lot, Adam! All sounds great
Please keep this list posted.
Cheers
Herbert
> To unsubscribe from this group and stop receiving emails from it, send an email to memento-dev+unsubscribe@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
>
>
> --
> Herbert Van de Sompel
> Digital Library Research & Prototyping
> Los Alamos National Laboratory, Research Library
> http://public.lanl.gov/herbertv/
> http://orcid.org/0000-0002-0715-6126
>
> ==
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups "Memento Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to memento-dev+unsubscribe@googlegroups.com.
---
You received this message because you are subscribed to the Google Groups "Memento Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to memento-dev+unsubscribe@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages