specifying a dependency of an operator on another operator

[Pavel Mores]

 
Hi all,
 
I think I need some help considering operator dependencies.  I work on
Sandboxed Containers Operator[*] (it uses Kata to run pods within VMs).
We added a validating webhook to it by generating it with operator-sdk.
The operator-sdk-generated webhook relies on cert-manager to manage the
webhook's certificate so our operator now depends on cert-manager.
 
I'd like to specify cert-manager as a dependency of Sandboxed Container
Operator.  To that end, I added the following to our
ClusterServiceVersion:
 
spec:
    required:
    - kind: Certificate
      name: certificates.cert-manager.io
      version: v1alpha2
 
This does seem to pull in cert-manager when I install our operator via
OperatorHub, however, there seems to be some interaction between our
and cert-manager's installModes.
 
At the moment, we support OwnNamespace and SingleNamespace while
cert-manager only supports AllNamespaces.  With this config, an attemp
to install our operator fails because cert-manager enters a Failed state
with "Failed: OwnNamespace InstallModeType not supported, cannot
configure to watch own namespace".
 
If I remove the support for the offending OwnNamespace installMode from
our CSV, now OpenShift console won't let me install because you have to
support at least one of a) running in one namespace, b) running in all
namespaces.  The check is performed by testing that at least one of
OwnNamespace and AllNamespacess is supported (I found this in the
OpenShift console code).
 
So, not being able to support OwnNamespace due to cert-manager
requirements, if I turn on AllNamespaces in our CSV at this point,
installation seems to conclude successfully however our operator's
controller manager ends up stuck in CrashLoopBackOff with an error
message that suggests it cannot find a certificate.  I think it's worth
pointing out here that although both operators are installed,
cert-manager runs in our operator's namespace for some reason.  I
suspect this might a problem because if you inspect cert-manager's
installation .yaml[**] it looks like cert-manager relies on running in
its own namespace (called 'cert-manager').
 
I'm looking for a way to specify cert-manager as a dependency so that it
runs in its own namespace (note that if I give up on using the
dependency mechanism and just install cert-manager manually and then our
operator, everything works).
 
Or, more broadly, seeing as operator-sdk's webhooks seem to rely on
cert-manager which is not a standard part of OpenShift clusters, I'm
probably not alone running into this problem - what is the
usual/expected way of dealing with this?
 
Thanks in advance,
 
    pvl
 
[*] https://github.com/openshift/sandboxed-containers-operator
[**] https://github.com/jetstack/cert-manager/releases/download/v1.4.0/cert-manager.yaml

[Alex Greene]

Good morning Pavel,

I recommend reviewing the OLM Webhook Documentation [1], specifically the information regarding OLM's generation of self-signed certificates for operators that include webhooks [2].

At a high level, when deploying operators that require webhooks, OLM will automatically generate a self signed certificate for your operator, therefore there is no need for you to specify a dependency on the cert-manager operator. This should resolve the issue you've hit.

There are plans for OLM to eventually integrate with certificate services [3], and if/when this change is made it will be done in a backwards compatible way.

[1]

https://olm.operatorframework.io/docs/advanced-tasks/adding-admission-and-conversion-webhooks

[2] https://olm.operatorframework.io/docs/advanced-tasks/adding-admission-and-conversion-webhooks/#certificate-authority-requirements
[3] https://github.com/operator-framework/operator-lifecycle-manager/issues/1805

--
You received this message because you are subscribed to the Google Groups "Operator Framework" group.
To unsubscribe from this group and stop receiving emails from it, send an email to operator-framew...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/operator-framework/c4cc9a3f-3884-4ada-b3dd-d6d2b6e1edf4n%40googlegroups.com.

--

Alexander Greene
He - Him - His

Senior Software Developer

C: 412-508-5479

IRC: agreene

[Daniel Messer]

As for the general question about installmodes - this is something the we are moving away from as part of the descoping exercise, in which all Operator just become global singleton's and we have more precise RBAC controls that define where they are available to be discovered by tenants, which tenants actually have usage rights and inversely, in which namespaces the operator can become active.

Our recommendation today is to only support AllNamespace mode in your Operator. That way dependencies will not run into the conflict you describe here.

To view this discussion on the web visit https://groups.google.com/d/msgid/operator-framework/CAOuJmZaDzRZj-th1%2BRwAAUuaotMC9TJodHFkJzA-SvwYs2K-ag%40mail.gmail.com.

--

Daniel Messer

Product Manager Operator Framework & Quay

Red Hat OpenShift

[Chris Johnson]

Sadly, the AllNamespace mode doesn't satisfy a large customer requirement of ours.  Many have concerns over operators having any Cluster permissions, which AllNamespace mode uses (it promotes all permissions in the CSV to ClusterRole and ClusterRolebindings.  Until the new "cluster mode / descoped mode" becomes available, we're requiring all of our (IBM) operators to support both AllNamespaces and OwnNamespace mode and strongly suggesting to customers to only install the operator once when using OwnNamespace mode.  For our suites of operators that all interoperate with one another we then require the customer to pick one of the modes for the entire stack and stick with it.

[Pavel Mores]

Hi! Thanks for your advice, I switched our certificate management from cert-manager to OLM by declaring the webhook in the ClusterServiceVersion and it worked!

[Pavel Mores]

So if I understand correctly, trying to solve the installModes conundrum I described in my initial post doesn't really seem worth the trouble anymore - good! :-)

[Pavel Mores]

Hi again! I have a follow-up question. While the certificate management by OLM works perfectly when our operator is installed as a bundle, we're noticing that installation via operator-sdk's 'make deploy' doesn't work anymore.  I'm assuming this is because 'make deploy' doesn't know about OLM and doesn't seem to create OLM-specific resources that let OLM understand that it should take over the cert management for the operator being installed.  At any rate, 'make deploy' now results in a non-functional operator installation due to certificate not found.

Is there a way to fix 'make deploy' under the circumstances?  More specifically, I'd like to understand what a bundle installation does apart from just installing the bare-bones operator (which is what 'make deploy' seems to do) in order to find out if there's a way to emulate the extras for 'make deploy'.  I know that a bundle also installs a CSV and (probably?) creates a Subscription but is that all?  I tried to install our CSV on a cluster before installing the operator proper with 'make deploy' but this didn't fix the problem.

I'm not even sure if this is the right forum to ask as the problem seems to exist somehow between OLM and operator-sdk so feel free to refer me elsewhere if appropriate.

Thanks in advance!

pvl