OLM : Consumption of an operator from Operatorhub.io Catalog behind Proxy

[Loic Fouray]

Hi,

I have a problem and i would like your help please :)

Indeed from a Kubernetes native without direct access to web, i have deployed OLM. I used my docker Registry Proxy (Ie Artifactory) to access to quay.io registry (for OLM images). It works.

But, when i want deploy an Operator from OperatorHub.io Catalog, the pull of image Operator (ie elastic for example) is KO because i don't access directly to quay.io registry and I would like to avoid repackaging all operators with my docker registry proxy.

I have a proxy and i would like to be able to install the operators by pulling the images by this proxy.
I try to integrate my Proxy configuration in the subscription declaration but without success :

apiVersion: operators.coreos.com/v1alpha1

kind: Subscription

metadata:

  name: eck

spec:

  config:

     env:

     - name: HTTP_PROXY

       value: http://cntlm:3128

     - name: http_proxy

       value: http://cntlm:3128

     - name: HTTPS_PROXY

       value: http://cntlm:3128

     - name: https_proxy

       value: http://cntlm:3128

     #- name: NO_PROXY

     # value: test

  channel: stable

  installPlanApproval: Automatic

  name: elastic-cloud-eck

  source: operatorhubio-catalog

  sourceNamespace: catalog-operator

  startingCSV: elastic-cloud-eck.v1.2.1

Also, i try to integrate Proxy configuration into Deployment Catalog-operator (from OLM Namespace), but it doesn't seem taken into account.

Have you an idea?

Thanks

Loïc

My Environment

• operator-lifecycle-manager version: 0.16.1 or v0.17.0
  

• Kubernetes version information: 1.18

[Kevin Rizza]

As far as I know, there is no upstream tool that can solve this problem generically for you. At a high level, the problems are:

The operator manifests themselves define the image/images that the operator actually runs to start its controllers. You would need a way to find out what those images are, copy them into a registry that the cluster can interact with, and update the operator bundles themselves to point to the accessible registry.

Many / most operators also have a set of images that they deploy and run as part of interacting with their APIs. The operator-framework has defined the concept of "related images" that some operators have opted into, whereby the images they use as part of runtime are set as environment variables that are injected into the operator itself when it is installed by OLM. Those references, if the operator actually implements them (rather than just hardcoding the image references in its source code) would also need to be copied and updated to point to the correct registry.

Lastly, OLM actually references these operator manifests by a pointer to a bundle image (inside the operator catalog) that it pulls down onto the cluster at runtime to install the operator. To get that to work, you would need to modify *those images* for the above two steps (by rebuilding them somehow) and copy those into the accessible registry, then build a new catalog image that points to those bundles.

Needless to say, there's a lot of internal moving bits here that make creating a simple image mirroring process maintainable (especially given that some of this relies on each operator following the correct pattern), and a lot of image rebuilds and modifying manifests. OKD and OpenShift have handwaved some of those problems by using ImageContentSourcePolicies and building some mirroring tools into their related toolkits, but without that API it's a very nontrivial problem.

It does seem like there is some appetite in the kube community for attempting to solve the proxied/offline/disconnected cluster problem in general, and at some point when that happens it would probably make sense for the operator framework to build on top of whatever solution happens there. But until it does in a first class way, there's just not really a trivial way to make this work for OLM. You *might* be able to configure some of this to work with very specific operators using proxies to force the container runtime to pull the right set of images, but there are a lot of caveats and it's probably not something that OLM would attempt to integrate with at this time.

Thanks,

Kevin Rizza

OLM Maintainer

--
You received this message because you are subscribed to the Google Groups "Operator Framework" group.
To unsubscribe from this group and stop receiving emails from it, send an email to operator-framew...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/operator-framework/7a25bc24-cdf4-4f67-8496-6c35be32b1e1n%40googlegroups.com.

[Loic Fouray]

Hi,

Thanks Kevin four your reply.

Currently Openshift give the possibiliy to integrate the Proxy configuration into the subscription : https://docs.openshift.com/container-platform/4.6/operators/admin/olm-configuring-proxy-support.html

Why does it work for Openshift and not for kubernetes? Is It functional because to ImageContentSourcePolicies feature used by Openshift?

Thanks

Loïc

[Kevin Rizza]

Keep in mind that the proxy configuration is for the context of the operator itself. You would need to set up a proxy in a very specific way on the *node* that the container runtime is running on to actually pull the right set of images -- OLM is just creating a deployment for the operator, so the container runtime itself would have to be redirected to pull from the right registry source. That proxy config won't accomplish that.

To view this discussion on the web visit https://groups.google.com/d/msgid/operator-framework/d947528f-e3b7-40bf-9809-e9ee635e50c3n%40googlegroups.com.

[Loic Fouray]

Yes, after that my operator will be deployed, i should update the images registry used by default in the CR. For exemple, with eck operator, it's possible to overload.  Without repackaging an operator with his bundle, is it possible for an admin cluster to enrich, overload the CR by default that will exposed to customers?  

But before, i would deploy my operator.

If I repeat your explanation:

"Lastly, OLM actually references these operator manifests by a pointer to a bundle image (inside the operator catalog) that it pulls down onto the cluster at runtime to install the operator. To get that to work, you would need to modify *those images* for the above two steps (by rebuilding them somehow) and copy those into the accessible registry, then build a new catalog image that points to those bundles."

I think that my problem is on this point. After the subscription created, i have a pod (created by OLM) with "Init:ImagePullBackOff" status. This pod try to extract eck bundle and to pull image

The pull  quay.io/operatorhubio/elastic-cloud-eck:v1.2.1 image is KO

If I understood correctly,  I should build a new catalog image by updating the registry of bundle images. That's right? No other solutions?

Thanks

Loïc