Concerns about Kubeflow 1.3 Deployment Strategy

[Bernd Verst]

What are we doing about the fact that Kubeflow cannot be deployed with the latest kustomize 3.X or any 4.x versions?

The latest versions that seem to work for me are 3.9.1 and 3.8.10.

On the one hand we ask users to use standard tooling (instead of kfctl in the past) which is good. On the other hand we don't support recent versions and hence require installing other kustomize binaries.

Either this is a 1.3 release blocker or kustomize is the wrong tool due to breaking changes. I can already foresee many folks filing issues about deployment problems due to the fact they are using newer versions of kustomize. I certainly tend to run the latest stable releases of all tools myself.

I don't think distribution owners should need to hide or work around these issues and deal with the support aftermath.

Thanks!
Bernd

Bernd Verst | bve...@gmail.com | LinkedIn | @berndverst

[Bernd Verst]

I have a potential fix! Though more eyes will be good.

I did some digging and it seems that at some point kustomize stopped supporting pointers in YAML (TODO: someone to open an upstream kustomize issue). The Knative manifests seem to be using these in a few places. I made a PR to remove them: https://github.com/kubeflow/manifests/pull/1806

With that change the latest kustomize 4.0.5 seems to be working for me, at least for the example deployment kustomization.yaml Yannis provided. 🎉 

Here is the tracking issue for support of newer kustomize versions: Upgrade to use Kustomize v4+ · Issue #1797 · kubeflow/manifests (github.com)

Bernd

Bernd Verst | bve...@gmail.com | LinkedIn | @berndverst