Fwd: jep/213 "Configuration Storage API" vs SideCar

[nicolas de loof]

(Forwarded from jenkins-dev mailing list)

Hi,

I've read JEP-213 proposal with lots of interest, 

IIUC there's no plan to change the configuration granularity (we still will have one configuration entry per legacy xml file) neither do we plan to change the data being stored here (plain component XStream dump, no plan to filter default values)

Then, I wonder we could adopt a simpler (?) approach relying on a sidecar configuration container.

i.e a typical k8s pattern for legacy applications is to deploy a sidecar to act as http proxy, and (for sample) add support for https or other improvements without changing a single bit of the existing codebase.

doing so we could offer Cloud Native configuration storage for Jenkins 

(illustration from Oreilly's "Designing-Distributed-Systems : PatternsParadigms")

Benefit I can see doing so is that we just don't need to wait for a new jenkins-core release, this can be adopted without delay.

wdyt ?

(Jesse's answer and my inlined answers)

> If I understand your proposal correctly, you suggest that a sidecar
> container fetch cloud-native configuration (say, K8S CRDs) as an
> initialization task and save it into `$JENKINS_HOME/**/*.xml` in
> traditional XStream format that unpatched Jenkins cores can read. 

Right. a configMap can be directly mounted as a volume, but this could also support other storages if this makes sense.

Using an InitContainer we can ensure this "configuration pull" operation completes before the master is started.

> This is certainly something to consider seriously, as it would involve much
> compatibility concerns that existing proposals, and the volume of
> configuration files is small enough that we are not concerned about
> the I/O implications of mirroring solutions (unlike other cloud-native
> storage problems such as artifacts and logs).

Yes indeed, my point considering this option is that afaict JEP-213 does not introduce a comparable compatibility break as other cloud native storage effort did. 

> Then the issue becomes synchronization of dynamic changes. For cloud →
> Jenkins we can assume that detecting changes on the cloud side is easy
> enough, and we can overwrite XML files on the shared drive, but how to
> notify Jenkins that we should reload just that file? Currently there
> is no such general system in Jenkins: you can make (REST) API calls to
> reload a particular `AbstractItem`, or the system (`Jenkins`) as a
> whole, but not necessary a single `GlobalConfiguration` or the like.

Do we want to support such partial configuration updates from cloud  --> jenkins ?

A possible approach is to rely on xml config file path for some known components, typically job/*/config.xml can be translated into an AbstractItem name to reload by a REST call. But this only will cover a subset of jenkins configuration. For any other change we can offer a full jenkins reload, in doubt.

> For the other direction, the sidecar could watch for changes to files
> it manages and mirror them to cloud storage. Can a K8S container rely
> on the existence of inotify, or does it need to poll? Or can we just
> look for changes after the pod shuts down?

We indeed can have inotify watchers on sidecar to detect configuration updates by jenkins UI and update the cloud configuration accordingly by pushing the modified xml content.

--

Nicolas De Loof

[Jesse Glick]

On Mon, Oct 1, 2018 at 11:05 AM nicolas de loof
<nicolas...@gmail.com> wrote:
> afaict JEP-213 does not introduce a comparable compatibility break as other cloud native storage effort did.

Unclear. To my knowledge no real compatibility analysis has been done yet.

Also there are a number of files in `$JENKINS_HOME` which are
configuration or state yet do not go through `XmlFile`, such as
`FileBoolean`s. I think these would be easier to handle under your
proposal, since we would just patch the sidecar to read/write such
files and translate to some cloud-native storage TBD, without needing
to introduce yet more SPIs into `jenkins-core`.

> job/*/config.xml can be translated into an AbstractItem name to reload by a REST call. But this only will cover a subset of jenkins configuration. For any other change we can offer a full jenkins reload, in doubt.

Yes that may be enough. Given JEP-201 I do not really expect
cloud-native users to be synchronizing in this direction anyway:
production configuration changes would be made by committing YAML
modifications. (You might object that a refinement of JEP-201 should
make the Jenkins config UI view-only, which is true, but this does not
obviate the need to synchronize from Jenkins to external storage: some
`$JENKINS_HOME` files or parts of files are state, not configuration,
and are written not as part of a config form save gesture. Admittedly
a lot of these cases correspond to state which we would tolerate
forgetting between sessions, depending on the workflows you want to
support.)

I am curious as to the reaction from Oleg (JEP-213 sponsor) as well as
Alex Nordlund, Mathieu Coavoux, and James Strachan (authors of some
existing prototypes in this area).

[Oleg Nenashev]

JEP-213 in the current state is designed for "classic Jenkins" architectures. If we talk only about Cloud Native Jenkins with single-shot agents and configuration-as-code, this story would require a different design.Maybe hust YAGNI, except some compatibility layers for plugins using XmlFile.

Compatibility... There WILL BE compatibility issues for plugins using java.io.File API in combination with XmlFile. https://github.com/jenkinsci/jep/blob/master/jep/213/README.adoc#new-storage-implementations . We have not performed investigation yet, but there are likely such ones. Also, some plugins will become unusable performance killers if they do not get updates for new APIs (e.g. Disk Usage plugin)

BR, Oleg

--
You received this message because you are subscribed to the Google Groups "Jenkins Cloud Native SIG" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkins-cloud-nati...@googlegroups.com.
To post to this group, send email to jenkins-clou...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkins-cloud-native-sig/CANfRfr1n6pD-%3DO0Gx2cVzdK4N36uOjh5UZvgF8XpYfQKnjeTgA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Jesse Glick]

On Mon, Oct 1, 2018 at 12:40 PM Oleg Nenashev <o.v.ne...@gmail.com> wrote:
> JEP-213 in the current state is designed for "classic Jenkins" architectures.

And that is the mode that Nicolas’ proposal addresses I think: Jenkins
as a long-running service.

> If we talk only about Cloud Native Jenkins with single-shot agents and configuration-as-code, this story would require a different design.

In ephemeral mode we do not need anything at all. JEP-201 suffices.

[nicolas de loof]

Le lun. 1 oct. 2018 à 18:49, Jesse Glick <jgl...@cloudbees.com> a écrit :

On Mon, Oct 1, 2018 at 12:40 PM Oleg Nenashev <o.v.ne...@gmail.com> wrote:
> JEP-213 in the current state is designed for "classic Jenkins" architectures.

And that is the mode that Nicolas’ proposal addresses I think: Jenkins
as a long-running service.

Right. "Sidecar container" is the noun I used here by intention to reflect k8s deployment pattern, but the exact same principle could apply on any classic jenkins - this is just another process running aside jenkins, nothing much (+ some namespace magic)

 

> If we talk only about Cloud Native Jenkins with single-shot agents and configuration-as-code, this story would require a different design.

In ephemeral mode we do not need anything at all. JEP-201 suffices.

--
You received this message because you are subscribed to the Google Groups "Jenkins Cloud Native SIG" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkins-cloud-nati...@googlegroups.com.
To post to this group, send email to jenkins-clou...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkins-cloud-native-sig/CANfRfr3tyJFx0ek_UhC%3D-wD5Q1_R-H-FNCYvBQZ3QAA2EEp_Pw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Nicolas De Loof

[Alex Nordlund]

On Mon, Oct 1, 2018 at 6:26 PM Jesse Glick <jgl...@cloudbees.com> wrote:

On Mon, Oct 1, 2018 at 11:05 AM nicolas de loof
<nicolas...@gmail.com> wrote:
> afaict JEP-213 does not introduce a comparable compatibility break as other cloud native storage effort did.

Unclear. To my knowledge no real compatibility analysis has been done yet.

No real compatibility analysis has been done yet, but some of the awkwardness in the current API is from trying very hard not to break compatibility.