[API Specification] Design goals for fixity spec?

[Benjamin Armintor]

Reading the notes for the last committer call, I see that there was an open question about whether the result of a fixity check should be persisted in the repository. The work that has been done to date on the fixity spec has presumed that, but without a documented design goal for the service I don't know how to judge the necessity of that.

There's an argument to be made that all we ought to require is that a LDP-S support:

1. LDP-NR with description, so that there's a resource to attach fixity info to

2. Want-Digest

and that everything else is a client application (eg, the client application can decide where it will store the original checksum in the repository, issue HEAD with Want-Digest on its own schedule, and do what it will with match data).  Even the versioning spec is optional if the client only cares about the most recent state.

This has the considerable advantage of being a minimal departure from LDP 1 (and very easy to implement), but I think it also means the API Spec doesn't have a Fixity Spec- the requirement of Want-Digest can just go into CRUD/LDP refinement, and fixity becomes a use case. It also means clarifying that fixity assertions stored in the repo don't have any endorsement by the repo: the client is just inserting assertions in the user data space. That seems like it would also relocate some responsibilities attached to TDR verification to the application layer.

If we go in that direction, it may also be worth considering whether we should propose an Expect header token to LDP-Next for expected digest values.

- Ben

[Esmé Cowles]

I like that direction: it gives a good separation of responsibility between storing the state of the binary vs. checking that state on some schedule.

Though I do wonder if there's a good way for a HEAD request to include the expected checksum, and the ability for the repository to generate a message and/or audit event if it doesn't match?

-Esmé

> --
> You received this message because you are subscribed to the Google Groups "Fedora Tech" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to fedora-tech...@googlegroups.com.
> To post to this group, send email to fedor...@googlegroups.com.
> Visit this group at https://groups.google.com/group/fedora-tech.
> For more options, visit https://groups.google.com/d/optout.

[A. Soroka]

The result of a fixity calculation may not be a checksum at all. It may be something that couldn't possibly fit in an HTTP header.

---
A. Soroka
The University of Virginia Library

[Esmé Cowles]

Fair enough — those will need to have some other mechanism.

But for the resources that do have checksums, I think a good pattern would be sending a HEAD request with one or more algorithms in the Want-Digest header, and optionally the expected values in the Digest header. If the expected values are supplied and don't match the calculated checksums, the the repository should generate a message and/or audit event to record that without requiring any extra action by the client.

-Esmé

[A. Soroka]

I don't think we can do exactly that: Digest isn't our header to redefine. In any event, I think this suggestion takes us right back in the territory of predicting what users want. (I'm not saying your prediction is wrong, Esmé! :grin:)

When I look in https://wiki.duraspace.org/display/FF/Use+Cases , I can't find any cases that seem to me to relate specifically to fixity, but over and over in different venues we hear that fixity is important to users. I wonder if we need to get a better handle on how sites use and want to use fixity?

---
A. Soroka
The University of Virginia Library

[Andrew Woods]

Thanks, Ben, for responding to today's Fedora Tech Meeting discussion, as it relates to a Fixity checking capability:

https://wiki.duraspace.org/display/FF/2016-12-08+-+Fedora+Tech+Meeting

This is exactly one of the things we have been hoping to get out of the Fedora specification effort: Debate and understanding of core Fedora repository services.

The assumption has been that "fixity checking" is a core responsibility of a Fedora repository. The discussion on today's tech call was focused on the interaction model for invoking the fixity checking service and whether information describing the invocation of such a service should be automatically persisted in the repository or not.

You are now raising the possibility that "fixity checking" may not be the responsibility of a Fedora repository.

I am going to make two suggestions,

1. We keep the draft Fedora API specification as-is until we get broader community input on questions around the Fixity checking service.

- http://fedora.info/spec/

2. We invite broader community input on this topic here in the mailing-list, as well as in a "special topic" call dedicated to raising and understanding use cases.
- Here is a Doodle poll for scheduling this call to take place in January: http://doodle.com/poll/r4dhd5hc5dfmcqzy

If you have an interest in the Fixity checking service in Fedora, please make sure your voice is heard.

Regards,

Andrew

>>>> To unsubscribe from this group and stop receiving emails from it, send an email to fedora-tech+unsubscribe@googlegroups.com.

>>>> To post to this group, send email to fedor...@googlegroups.com.
>>>> Visit this group at https://groups.google.com/group/fedora-tech.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups "Fedora Tech" group.

>>> To unsubscribe from this group and stop receiving emails from it, send an email to fedora-tech+unsubscribe@googlegroups.com.

>>> To post to this group, send email to fedor...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/fedora-tech.
>>> For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google Groups "Fedora Tech" group.

>> To unsubscribe from this group and stop receiving emails from it, send an email to fedora-tech+unsubscribe@googlegroups.com.

>> To post to this group, send email to fedor...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/fedora-tech.
>> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups "Fedora Tech" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to fedora-tech+unsubscribe@googlegroups.com.

> To post to this group, send email to fedor...@googlegroups.com.
> Visit this group at https://groups.google.com/group/fedora-tech.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Fedora Tech" group.

To unsubscribe from this group and stop receiving emails from it, send an email to fedora-tech+unsubscribe@googlegroups.com.

[Andrew Woods]

Hello All,

Thanks for those who responded to the poll:
http://doodle.com/poll/r4dhd5hc5dfmcqzy

We will hold a special topic call on January 9th @10am ET to define a proposal to the Fedora community for specification of the Fixity Checking Service.

Meeting page:
https://wiki.duraspace.org/display/FF/2017-01-09+Fixity+Service+Special+Topic+meeting

Agenda:
1. Is Fixity Checking a core repository service?
2. If yes to #1, what is the interaction model?
3. If yes to #1, should the result of executing the Fixity Checking service be specified as being persisted in the repository?
4. If yes to #1, should repository messages be emitted upon the completion of executing the Fixity Checking service?

Time:
https://www.timeanddate.com/worldclock/fixedtime.html?msg=Fedora+Fixity+Checking+Service&iso=20170109T10&p1=179&ah=1

Call-in info:
- Dial-in Number: (712) 775-7035
- Participant Code: 479307#

Regards,

Andrew

[Andrew Woods]

Hello All,

This is a reminder to please join the Fixity Service special topic meeting this coming Monday if you are interested in Fedora's contract for resource fixity checking.

https://wiki.duraspace.org/display/FF/2017-01-09+Fixity+Service+Special+Topic+meeting

Regards,

Andrew

[Benjamin Armintor]

Based on the feedback on the call today and this thread, I put together a quick summary of header-driven, HTTP/1.1-friendly approaches to folding a baseline byte-fixity verification into the CRUD spec. The draft is up at https://docs.google.com/document/d/1Nm-LFyUhCl-et4UsdePuMJ_Zu23Mot4FiEpEMke_JH4/edit.

I'll do what I can to refine the abstract; comments welcome; etc.

- Ben

ps happy new year

You received this message because you are subscribed to the Google Groups "Fedora Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fedora-community+unsubscribe@googlegroups.com.

[Andrew Woods]

Thank you to those who attended today's Fixity Special Topic meeting.

https://wiki.duraspace.org/display/FF/2017-01-09+Fixity+Service+Special+Topic+meeting

The recommended proposal for Fedora's specified fixity service can be summarized as follows:
- On ingest, Fedora will calculate a checksum
- Client will get an error response if it tries to ingest something with a user provided value that does not match the calculated value
** No guarantee that this will be stored, but a client can always choose to store it like any other info
- Client can provide a fixity digest again in a header (Expect, Want-Digest, etc. See [1]) on a GET or HEAD request and server will recalculate fixity to compare, returning a 4xx error response if they do not match

Thanks to Ben Armintor for drafting specification options [1] for supporting this interaction model.

While the draft [1] is in its Google-Doc form, please provide your questions or comments directly in that document. If consensus settles on one of these options, it will be moved to the Fedora API specification GitHub repository [2], after which point please compose your comments as GitHub issues.

And as always, feel free to respond directly to the fedora-community mailing list.

Regards,

Andrew

[1] https://docs.google.com/document/d/1Nm-LFyUhCl-et4UsdePuMJ_Zu23Mot4FiEpEMke_JH4/edit

[2] https://github.com/fcrepo/fcrepo-specification