Has anyone made an operator with Go submodules?

[jonathan...@gmail.com]

Has anyone made a Golang operator with a Go submodule? In this issue:

https://github.com/operator-framework/operator-sdk/issues/5583

we have a user that wants to declare their api directory that contains their CRD golang structs as a submodule for import purposes. This is breaking because now Operator-SDK can't see some of the golang types the CRD structs use from the base directory, and the commands can't be used from the sub directory because SDK can't see the PROJECT file.

We can't really think of a good way to fix this without causing additional problems. Has anyone made an operator with Go submodules and experienced anything similar or found any workarounds?

-Jonathan Berkhahn

Operator-SDK

[Nick Carboni]

I'm one of the maintainers on that project and would be happy to talk about our use case or help test out some possible fixes or workarounds.
Unfortunately what we're doing for now is:

• copy the PROJECT file into the api directory
  
• `operator-sdk generate kustomize manifests --apis-dir . --input-dir ../config/manifests --output-dir ../config/manifests`
• remove api/PROJECT and api/config

Nick

--
You received this message because you are subscribed to the Google Groups "Operator Framework" group.
To unsubscribe from this group and stop receiving emails from it, send an email to operator-framew...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/operator-framework/e29b5284-d557-464c-a969-a4f0025d7dc8n%40googlegroups.com.

[Camila Macedo]

Hi Nick, 

Assuming that you created the project with SDK and you changed its default layout; 

a) Could you share the link for your repo, so we can see the changes made in the default layout which caused this behaviour? Could you provide the code, so we can check and reproduce the issue?

b) Could you please let us know more about what motivates you to make these changes? What problems/scenarios lead you to make these changes? 

Cheers, 

CAMILA MACEDO

SR. SOFTWARE ENGINEER 

RED HAT Operator framework

Red Hat UK

She / Her / Hers

IM: cmacedo

I respect your work-life balance. Therefore there is no need to answer this email out of your office hours.

To view this discussion on the web visit https://groups.google.com/d/msgid/operator-framework/CAL3emWYVrxyPcGx_weuTNG3Y-XT_jJEVfMrVodWWc1LHeDxeYg%40mail.gmail.com.

[Camila Macedo]

Hi Michael, 

Thank you for the info provided and all your input to help us try to check it out.

Following my thoughts and POV about the scenario and request. 

First, let's try to understand the use case:

I want to create a go module for my apis so that I can import it.

Now let's try to mock this scenario: 

Let's use the Memcached sample: https://github.com/operator-framework/operator-sdk/tree/master/testdata/go/v3/memcached-operator.

a) Run `go mod init` inside the api directory

b) Run `make bundle`

Then, we will check the error:

bin/controller-gen "crd:trivialVersions=true,preserveUnknownFields=false" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases

-: go build github.com/example/memcached-operator/api/v1alpha1: no Go files in

Error: not all generators ran successfully

run `controller-gen crd:trivialVersions=true,preserveUnknownFields=false rbac:roleName=manager-role webhook paths=./... output:crd:artifacts:config=config/crd/bases -w` to see all available markers, or `controller-gen crd:trivialVersions=true,preserveUnknownFields=false rbac:roleName=manager-role webhook paths=./... output:crd:artifacts:config=config/crd/bases -h` for usage

make: *** [Makefile:85: manifests] Error 1

That means the controller-gen (https://github.com/kubernetes-sigs/controller-tools/tree/master/cmd/controller-gen) is raising an error and is not accepting that.

So, the first point here is we check if controller-gen could be able to work and allow it. 

That shows all that we need to do here to address your scenario. 

Now, let's think about your specific scenario and request: 

By looking at the source code:

• The PROJECT file: https://github.com/openshift/assisted-service/blob/master/PROJECT (we can check that you only scaffold one API)
• However, by looking at the repo: https://github.com/openshift/assisted-service/tree/master/api (we can check that you have apis from different groups. Am I right?) 

TL'DR: Then, that means you ought to use the following command to enable the support to multi-groups: operator-sdk edit --multigroup=true so that the tool could work with a different layout. That would scaffold dif paths to allow you to do that. For further information, see https://book.kubebuilder.io/migration/multi-group.html.

• You can check a sample with the layout here: https://github.com/kubernetes-sigs/kubebuilder/tree/master/testdata/project-v3-multigroup.
• To check the diff, you can compare with: https://github.com/kubernetes-sigs/kubebuilder/tree/master/testdata/project-v3.

By looking at the errors faced:

Is SDK or its command complaining about what you have a Go Mod inside of the api directory? NO

So, the problem here does not seem related to the use or not submodules at all, see:

Error: Error generating kustomize files: error getting ClusterServiceVersion base: error generating ClusterServiceVersion definitions metadata: one or more API packages had type errors

That occurs because the command could not generate correctly the CSV based on the config/ data.

It probably is caused by the fact that we do not have all APIs scaffolded in your project using the tool

So, the specific manifests used, for example, to track the apis that the project has, are not updated, and we cannot generate the CSV.

See, e.g., https://github.com/operator-framework/operator-sdk/blob/master/testdata/go/v3/memcached-operator/config/crd/kustomization.yaml#L1-L6.

Error "unable to read "PROJECT" file: open PROJECT: no such file or directory."

Here the problem is that the tool cannot look for the PROJECT file in the expected path.

In this way, this problem shows to be more related to the fact that you did not use the tool to do all scaffolds, to define that your layout should support multi groups properly and deviated too much from its default layout. However, you still want to take advantage of the tool helpers. Thus, you still want to use it to generate the bundle and are looking for a way to accommodate it with all changes made by skipping the PROJECT file or informing another path for it to move forward with the command and not face the issue.

Please, ensure that you check: https://sdk.operatorframework.io/docs/faqs/#can-i-customize-the-projects-initialized-with-operator-sdk 

Therefore, regards your request/RFE:

As described above, to address your need, I do not think the change should be done in SDK.

To support work with Go modules and generate the files, we need to change controller-gen. 

However, see that:

• The info of your PROJECT file is not correct and does not match with your project since you have not used the tool to do scaffolds.
• The info in the config file also shows missing the required data since you did not use the tool. So that does not seem that your bundle would be appropriately built as we described above.

So, before we begin to think in solving the issues by adding flags that just bypass some internal code tool checks and pre-requirements (IMHO), we need first to think and understand: 

• How/when/why do we use the PROJECT file to generate the bundle? 
• • Is for we look at the correct paths for the apis? For example, the path changes when we have multigroup: true (supports multigroup). 
  • Is for us know if the layout is from go/v2, go/v3, ansible/v1, etc. plugins?
• Could we ensure that the bundle is generated correctly without the info provided? 
• If we need the info provided in the PROJECT file for this operation: could we ignore it by making valid assumptions? Should it not be better than the flag? Should we warn the uses in this scenario?  

Also, in your case, you make the command work without raising issues when you move the PROJECT file to the inside of the apis dir. However, that does not necessarily mean that the problem faced with the command is sorted out adequately, that the tool is generating the bundle with all data correctly, etc., well as expected. 

I hope that can help us out.

I am adding this info in the issue as well. 

Thank you for your attention and time. 

CAMILA MACEDO

SR. SOFTWARE ENGINEER 

RED HAT Operator framework

Red Hat UK

She / Her / Hers

IM: cmacedo

I respect your work-life balance. Therefore there is no need to answer this email outside your office hours.

On Tue, Mar 29, 2022 at 11:56 AM Michael Filanov <mfil...@redhat.com> wrote:

Hey,

The repo: https://github.com/openshift/assisted-service
The generation script:  https://github.com/openshift/assisted-service/blob/9a8432287116458e0fbafdfc08311c06134a6450/hack/generate.sh#L133-L175

The main issue with the import from between different projects.
In the past we had to import from a different project that had at lest 10 replaces in their go.mod file, sadly go modules didn't like that and i had to do a lot of tries and replaces in order to fix a conflict between the required packages versions. Because of those changes every other project that imported from our project had the same issue.
So eventually we fixed the original project and reduced the replaces in our repo as well, but still we have few replaces that we need to have.
So for a better use we separate the API into a different module, so it's really easy to import the api without getting all the other requirements in the project. 

Adding it to the ticket as well

[Paul Richardson]

Hey Nick,

Yes. The camel-k project [1] has done this since we wanted to share our API separately. To get the API compiled and the bundle generation working, we did have to change the 'standard' Makefile configuration [2] & [3], which requires temporarily copying the api directories to the root directory in order to generate the CRDs then utilising the --apis-dir switch on `generator kustomize manifests`. Its not the most readable code but it has now been quite effective for a few releases and operator-sdk versions.

Regards

Paul

[1] https://github.com/apache/camel-k
[2] https://github.com/apache/camel-k/blob/main/script/Makefile#L451
[3] https://github.com/apache/camel-k/blob/main/script/Makefile#L491

---
Paul Richardson
* p.g.ric...@phantomjinx.co.uk
* Ty-Carreg Barn, Penyrheol, Pontypool, Torfaen. NP4 5XZ
* (home) 01495 762483
* (mob) 07980 869490
* @enigmajx

"There is no operating system on the planet that can withstand a determined assault from a clueless user."