Event hooks
9 views
Skip to first unread message
Stefano Cossu
Nov 3, 2014, 10:31:15 AM11/3/14
to fedor...@googlegroups.com
Hello,
Following the recent changes in the Fedora codebase, I have noticed an increased general interest toward a more compact application core focused on a minimal set of features. I am glad to see this, which means a greater focus on critical aspects such as stability, maintainability and durability.
However, most of the not explicitly supported features we have been building upon at AIC so far, some of which are not recommended in the 4.0 release, are critical for us and we want to make an effort to maintain these features as close as possible to the repository layer. These include content modeling, structural and data type validation, event-driven automation and custom connectors. So I have been thinking about how functionality can be added to a minimal Fedora system in the most manageable way.
One interesting project I have noticed in the early development stages of F4 are webhooks and other Event-driven APIs [1]. These could be an interesting starting point for event-based automation on post-commit events; other functionality may be based on pre-commit or pre-retrieval events or need synchronous operation, such as validation, custom UID minting, etc.
If some event hooks could be formalized either in the core codebase or in a separate add-on, and a set of synchronous and asynchronous event APIs were developed, would this be a good way for the community to build features on top of the minimal Fedora code that are not critical for Fedora itself, but can be critical for some implementations of it?
These APIs would be ideally either part of the core, or designed and developed under the Fedora committers' coordination, in order for them to be stable and easily maintainable by a larger group of committers.
I know this might not be the ideal time for this type of discussion, as all eyes are pointed at the imminent 4.0 release. But I'd like to just drop these ideas here until someone picks them up.
Following the recent changes in the Fedora codebase, I have noticed an increased general interest toward a more compact application core focused on a minimal set of features. I am glad to see this, which means a greater focus on critical aspects such as stability, maintainability and durability.
However, most of the not explicitly supported features we have been building upon at AIC so far, some of which are not recommended in the 4.0 release, are critical for us and we want to make an effort to maintain these features as close as possible to the repository layer. These include content modeling, structural and data type validation, event-driven automation and custom connectors. So I have been thinking about how functionality can be added to a minimal Fedora system in the most manageable way.
One interesting project I have noticed in the early development stages of F4 are webhooks and other Event-driven APIs [1]. These could be an interesting starting point for event-based automation on post-commit events; other functionality may be based on pre-commit or pre-retrieval events or need synchronous operation, such as validation, custom UID minting, etc.
If some event hooks could be formalized either in the core codebase or in a separate add-on, and a set of synchronous and asynchronous event APIs were developed, would this be a good way for the community to build features on top of the minimal Fedora code that are not critical for Fedora itself, but can be critical for some implementations of it?
These APIs would be ideally either part of the core, or designed and developed under the Fedora committers' coordination, in order for them to be stable and easily maintainable by a larger group of committers.
I know this might not be the ideal time for this type of discussion, as all eyes are pointed at the imminent 4.0 release. But I'd like to just drop these ideas here until someone picks them up.
aj...@virginia.edu
Nov 3, 2014, 11:39:10 AM11/3/14
to fedor...@googlegroups.com
Stefano--
I think it makes a lot of sense to try and map out how some of these (very common) needs can be met, and event-driven mechanisms are certainly part of the picture. For the sake of a clear discussion, I'd like us to distinguish between those kinds of extensions the functionality of which can be understood as triggered by the completion of repository workflows (e.g. indexing), and those kinds that
can only be understood as intervening in that workflow.
The first class of extensions has "connections" available now (in the form of JMS messaging). We can certainly make it possible to have more varieties of connection (e.g. web hooks).
The second class is a little more difficult. For example, we could (and sometimes do!) mean several different things by "validation", but let's say that you want your repository to _reject_ mutations that would put it into an "invalid" state (defined by some appropriate means). That's an intervention in repository workflow, and right now, there aren't many defined "hooks" or "connections" available for that kind of purpose. I think the only public or specified extension points of that kind are those concerned with authorization.
So, is one of the suggestions you are making that Fedora 4 offer some points inside repository workflow where extension apparatus can intervene to modify it?
---
A. Soroka
The University of Virginia Library
> --
> You received this message because you are subscribed to the Google Groups "Fedora Tech" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to fedora-tech...@googlegroups.com.
> To post to this group, send email to fedor...@googlegroups.com.
> Visit this group at http://groups.google.com/group/fedora-tech.
> For more options, visit https://groups.google.com/d/optout.
I think it makes a lot of sense to try and map out how some of these (very common) needs can be met, and event-driven mechanisms are certainly part of the picture. For the sake of a clear discussion, I'd like us to distinguish between those kinds of extensions the functionality of which can be understood as triggered by the completion of repository workflows (e.g. indexing), and those kinds that
can only be understood as intervening in that workflow.
The first class of extensions has "connections" available now (in the form of JMS messaging). We can certainly make it possible to have more varieties of connection (e.g. web hooks).
The second class is a little more difficult. For example, we could (and sometimes do!) mean several different things by "validation", but let's say that you want your repository to _reject_ mutations that would put it into an "invalid" state (defined by some appropriate means). That's an intervention in repository workflow, and right now, there aren't many defined "hooks" or "connections" available for that kind of purpose. I think the only public or specified extension points of that kind are those concerned with authorization.
So, is one of the suggestions you are making that Fedora 4 offer some points inside repository workflow where extension apparatus can intervene to modify it?
---
A. Soroka
The University of Virginia Library
> You received this message because you are subscribed to the Google Groups "Fedora Tech" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to fedora-tech...@googlegroups.com.
> To post to this group, send email to fedor...@googlegroups.com.
> Visit this group at http://groups.google.com/group/fedora-tech.
> For more options, visit https://groups.google.com/d/optout.
Stefano Cossu
Nov 3, 2014, 12:20:57 PM11/3/14
to fedor...@googlegroups.com
Adam,
Yes, that's what I was suggesting.
I guess asynchronous, post-* hooks are the "easy" part (the most complex part of it is perhaps agreeing on some standards, e.g. which events should be exposed and the metadata available for them).
"Blocking" hooks, i.e. event hooks that can hold the workflow and make it fail or succeed, and/or modify its output based on a set of conditions determined by internal or external services (an authorization provider, a structural validation framework, data type validation, the horoscope, etc.), need more thinking and work as you mentioned.
After these extension points are available we could open a discussion of which add-on hooks are most needed by the community and how we can tackle them in a coordinated way.
Yes, that's what I was suggesting.
I guess asynchronous, post-* hooks are the "easy" part (the most complex part of it is perhaps agreeing on some standards, e.g. which events should be exposed and the metadata available for them).
"Blocking" hooks, i.e. event hooks that can hold the workflow and make it fail or succeed, and/or modify its output based on a set of conditions determined by internal or external services (an authorization provider, a structural validation framework, data type validation, the horoscope, etc.), need more thinking and work as you mentioned.
After these extension points are available we could open a discussion of which add-on hooks are most needed by the community and how we can tackle them in a coordinated way.
Stefano Cossu
Director of Application Services, Collections
The Art Institute of Chicago
116 S. Michigan Ave.
Chicago, IL 60603
312-499-4026
Exorcise Your
Mind
Temptation: The Demons of James Ensor
November 23–January 25
The Art Institute of Chicago
Voted #1 museum in the world by TripAdvisor
Stefano Cossu
Nov 3, 2014, 12:22:57 PM11/3/14
to fedor...@googlegroups.com
Errata: instead of "add-on hooks" I
meant "add-on services".
Stefano Cossu
Director of Application Services, Collections
The Art Institute of Chicago
116 S. Michigan Ave.
Chicago, IL 60603
312-499-4026
Exorcise Your
Mind
Temptation: The Demons of James Ensor
November 23–January 25
The Art Institute of Chicago
Voted #1 museum in the world by TripAdvisor
Reply all
Reply to author
Forward
0 new messages