[Just to share]: If you distribute your Operators via OLM, then be aware of Operator Bundle size limits

170 views

Skip to first unread message

Camila Macedo

unread,

Feb 23, 2022, 5:01:42 PM2/23/22

to Operator Framework

If you distribute your Operators via OLM, then be aware of Operator Bundle size limits

Bundles have a size limitation because their manifests are used to create a configMap, and the Kubernetes API does not allow configMaps larger than ~1MB. However, from OLM version v0.19.0, these values were increased to ~4MB because we are compressing them. (More info[0]). This change to allow bigger bundles from OLM v0.19.0 only impacts the full bundle size. Any single manifest within the bundle, such as the CRD, will still make the bundle uninstallable if it exceeds the default file size limit on clusters (~1MB).

💁‍♀️ How/when the bundle size limit error can be faced?

You will see errors like the following one when you try to install the Operator bundle via OLM:

time="2022-02-02T01:43:02Z" level=error msg="File with size 1359056 exceeded 1048576 limit, aborting" file=/bundle/manifests/myfile.clusters.test.com_example.yaml

Error: error loading manifests from directory: file myfile.clusters.test.com_example.yaml bigger than total allowed limit

🚀 What are the released versions which have the bundle size limit increased?

OPM (https://github.com/operator-framework/operator-registry) : 1.17.5[1]
OLM (https://github.com/operator-framework/operator-lifecycle-manager) : v0.19.0[2]

NOTE: If you publish your solutions via the Community repository ( https://github.com/redhat-openshift-ecosystem/community-operators-prod ) to distribute them in the OpenShift catalog; this change is from OCP 4.9.

✨ What does it mean for you:

If you publish an Operator to be distributed via OLM versions before ( < ) v0.19.0[2] then, the bundle size cannot be bigger than ~1MB.
If you publish an Operator to be distributed via OLM versions from OLM ( >= ) v0.19.0[2] then, the bundle size cannot be bigger than ~4MB uncompressed.
Any single manifest within your Operator bundle cannot exceed the file size limit of ~1MB.

✨How Operator-SDK can help:

From the next future Operator-SDK[7] release (1.18.0), Operator Authors can check the size limit without installing the operator bundle with OLM via the validators by running:

$ operator-sdk bundle validate <bundle-path>

That means if your bundle size is bigger than ~4MB, the above check will fail. Also, this check will warn when the bundle size is close to this amount.

NOTE: Currently, the above BundleValidator[6] will only check the full bundle size and not the size of each manifest (e.g. CRD) to verify that they are smaller than ~1MB.

Please, feel free to contribute. Your help is very welcome.

💡How to keep the CRDs smaller

We can design Operator solutions that have many APIs(CRDs). In this way, you might want to review the design of your Operator solution to ensure that it is not hurting concepts such as; encapsulation, the single responsibility principle, and cohesion. Also, you might need to find some alternative options to achieve the same goal in another way. To better understand the APIs, see Groups and Versions and Kinds, oh my![7].

Following are some general guidelines that we hope help you out:

Do all definitions must be in a single API(CRD/CR), or could it make sense to be split in a better API design?
Are you documenting the API(CRD)? Could you be more concise and less verbose? (but surely we don't want to say to not document the APIs)
How many versions of the CRDs[8] are supported in your Operator? Could you deprecate and reduce the number of supported APIs versions? (In this case, the recommendation can be to have a clear plan for deprecation and removal of old CRDs versions when a new one gets added, see Kubernetes API change practices[9]).

Also, you might want to give a look at Kubernetes API conventions[10].

[0] - https://github.com/operator-framework/operator-registry/pull/685

[1] - https://github.com/operator-framework/operator-registry/releases/tag/v1.17.5

[2] - https://github.com/operator-framework/operator-lifecycle-manager/releases/tag/v0.19.0

[3] - https://redhat-connect.gitbook.io/certified-operator-guide/ocp-deployment/operator-metadata/bundle-directory/managing-openshift-versions

[4] - https://github.com/operator-framework/operator-sdk/pull/5552

[5] - https://github.com/operator-framework/api/blob/master/pkg/validation/internal/bundle.go#L129-L151

[6] - https://github.com/operator-framework/operator-sdk

[7] - https://book.kubebuilder.io/cronjob-tutorial/gvks.html

[8] - https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definition-versioning/#specify-multiple-versions

[9] - https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api_changes.md

10- https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md