jep/213 "Configuration Storage API" vs SideCar

[nicolas de loof]

Hi,

I've read JEP-213 proposal with lots of interest, 

IIUC there's no plan to change the configuration granularity (we still will have one configuration entry per legacy xml file) neither do we plan to change the data being stored here (plain component XStream dump, no plan to filter default values)

Then, I wonder we could adopt a simpler (?) approach relying on a sidecar configuration container.

i.e a typical k8s pattern for legacy applications is to deploy a sidecar to act as http proxy, and (for sample) add support for https or other improvements without changing a single bit of the existing codebase.

doing so we could offer Cloud Native configuration storage for Jenkins 

(illustration from Oreilly's "Designing-Distributed-Systems : PatternsParadigms")

Benefit I can see doing so is that we just don't need to wait for a new jenkins-core release, this can be adopted without delay.

wdyt ?

-- 

Nicolas De Loof

[Jesse Glick]

On Mon, Oct 1, 2018 at 6:51 AM nicolas de loof <nicolas...@gmail.com> wrote:
> we just don't need to wait for a new jenkins-core release, this can be adopted without delay.

If I understand your proposal correctly, you suggest that a sidecar
container fetch cloud-native configuration (say, K8S CRDs) as an
initialization task and save it into `$JENKINS_HOME/**/*.xml` in
traditional XStream format that unpatched Jenkins cores can read. This
is certainly something to consider seriously, as it would involve much
compatibility concerns that existing proposals, and the volume of
configuration files is small enough that we are not concerned about
the I/O implications of mirroring solutions (unlike other cloud-native
storage problems such as artifacts and logs).

Then the issue becomes synchronization of dynamic changes. For cloud →
Jenkins we can assume that detecting changes on the cloud side is easy
enough, and we can overwrite XML files on the shared drive, but how to
notify Jenkins that we should reload just that file? Currently there
is no such general system in Jenkins: you can make (REST) API calls to
reload a particular `AbstractItem`, or the system (`Jenkins`) as a
whole, but not necessary a single `GlobalConfiguration` or the like.

For the other direction, the sidecar could watch for changes to files
it manages and mirror them to cloud storage. Can a K8S container rely
on the existence of inotify, or does it need to poll? Or can we just
look for changes after the pod shuts down?

[Jesse Glick]

BTW could you move this thread to
jenkins-clou...@googlegroups.com which is probably the right
forum?