Namespaced vs Clusterscoped Operator Install ?

[Dane DeValcourt]

I understand that Operator needs to be installed and uses cluster scopes for:

customresourcedefinitions, clusterroles, and clusterrole bindings as those inherently are all cluster-scoped

However this PR:  https://github.com/ansible/awx-operator/pull/541 talks about moving from Clusterscoped to namespaced-scoped.

When I do an Operator install using 0.15.0, I am seeing failures related to all the things that are cluster scoped:

customresourcedefinition.apiextensions.k8s.io/awxbackups.awx.ansible.com created customresourcedefinition.apiextensions.k8s.io/awxrestores.awx.ansible.com created customresourcedefinition.apiextensions.k8s.io/awxs.awx.ansible.com created clusterrole.rbac.authorization.k8s.io/awx-operator-metrics-reader created clusterrole.rbac.authorization.k8s.io/awx-operator-proxy-role created clusterrolebinding.rbac.authorization.k8s.io/awx-operator-proxy-rolebinding created

Errors from Operator install:

make deploy

cd config/manager && /Users/ddevalco/Dropbox/escm/awx19_5/awx-operator/bin/kustomize edit set image controller=quay.io/ansible/awx-operator:0.15.0

cd config/default && /Users/ddevalco/Dropbox/escm/awx19_5/awx-operator/bin/kustomize edit set namespace escm-stg

/Users/ddevalco/Dropbox/escm/awx19_5/awx-operator/bin/kustomize build config/default | kubectl apply -f -

namespace/escm-stg unchanged

serviceaccount/awx-operator-controller-manager unchanged

configmap/awx-operator-awx-manager-config created

service/awx-operator-controller-manager-metrics-service created

deployment.apps/awx-operator-controller-manager created

Error from server (Forbidden): error when retrieving current configuration of:

Resource: "apiextensions.k8s.io/v1, Resource=customresourcedefinitions", GroupVersionKind: "apiextensions.k8s.io/v1, Kind=CustomResourceDefinition"

Name: "awxbackups.awx.ansible.com", Namespace: ""

from server for: "STDIN": customresourcedefinitions.apiextensions.k8s.io "awxbackups.awx.ansible.com" is forbidden: User "u-w4453qrfpa" cannot get resource "customresourcedefinitions" in API group "apiextensions.k8s.io" at the cluster scope

Error from server (Forbidden): error when retrieving current configuration of:

Resource: "apiextensions.k8s.io/v1, Resource=customresourcedefinitions", GroupVersionKind: "apiextensions.k8s.io/v1, Kind=CustomResourceDefinition"

Name: "awxrestores.awx.ansible.com", Namespace: ""

from server for: "STDIN": customresourcedefinitions.apiextensions.k8s.io "awxrestores.awx.ansible.com" is forbidden: User "u-w4453qrfpa" cannot get resource "customresourcedefinitions" in API group "apiextensions.k8s.io" at the cluster scope

Error from server (Forbidden): error when retrieving current configuration of:

Resource: "apiextensions.k8s.io/v1, Resource=customresourcedefinitions", GroupVersionKind: "apiextensions.k8s.io/v1, Kind=CustomResourceDefinition"

Name: "awxs.awx.ansible.com", Namespace: ""

from server for: "STDIN": customresourcedefinitions.apiextensions.k8s.io "awxs.awx.ansible.com" is forbidden: User "u-w4453qrfpa" cannot get resource "customresourcedefinitions" in API group "apiextensions.k8s.io" at the cluster scope

Error from server (Forbidden): error when creating "STDIN": roles.rbac.authorization.k8s.io "awx-operator-awx-manager-role" is forbidden: user "u-w4453qrfpa" (groups=["azuread_group://"system:authenticated" "system:cattle:authenticated"]) is attempting to grant RBAC permissions not currently held:

{APIGroups:[""], Resources:["events"], Verbs:["delete"]}

{APIGroups:[""], Resources:["pods/log"], Verbs:["create"]}

{APIGroups:[""], Resources:["rolebindings"], Verbs:["get" "list" "create" "delete" "patch" "update" "watch"]}

{APIGroups:[""], Resources:["roles"], Verbs:["get" "list" "create" "delete" "patch" "update" "watch"]}

{APIGroups:[""], Resources:["services/finalizers"], Verbs:["get" "list" "create" "delete" "patch" "update" "watch"]}

{APIGroups:["apps"], Resources:["deployments/finalizers"], ResourceNames:["awx-operator"], Verbs:["update"]}

{APIGroups:["apps"], Resources:["ingresses"], Verbs:["get" "list" "create" "delete" "patch" "update" "watch"]}

{APIGroups:["awx.ansible.com"], Resources:["*"], Verbs:["*"]}

{APIGroups:["awx.ansible.com"], Resources:["awxbackups"], Verbs:["*"]}

{APIGroups:["awx.ansible.com"], Resources:["awxrestores"], Verbs:["*"]}

{APIGroups:["networking.k8s.io"], Resources:["daemonsets"], Verbs:["get" "list" "create" "delete" "patch" "update" "watch"]}

{APIGroups:["networking.k8s.io"], Resources:["deployments"], Verbs:["get" "list" "create" "delete" "patch" "update" "watch"]}

{APIGroups:["networking.k8s.io"], Resources:["replicasets"], Verbs:["get" "list" "create" "delete" "patch" "update" "watch"]}

{APIGroups:["networking.k8s.io"], Resources:["statefulsets"], Verbs:["get" "list" "create" "delete" "patch" "update" "watch"]}

{APIGroups:["rbac.authorization.k8s.io"], Resources:["configmaps"], Verbs:["get" "list" "create" "delete" "patch" "update" "watch"]}

{APIGroups:["rbac.authorization.k8s.io"], Resources:["endpoints"], Verbs:["get" "list" "create" "delete" "patch" "update" "watch"]}

{APIGroups:["rbac.authorization.k8s.io"], Resources:["events"], Verbs:["get" "list" "delete" "watch"]}

{APIGroups:["rbac.authorization.k8s.io"], Resources:["persistentvolumeclaims"], Verbs:["get" "list" "create" "delete" "patch" "update" "watch"]}

{APIGroups:["rbac.authorization.k8s.io"], Resources:["pods"], Verbs:["get" "list" "create" "delete" "patch" "update" "watch"]}

{APIGroups:["rbac.authorization.k8s.io"], Resources:["secrets"], Verbs:["get" "list" "create" "delete" "patch" "update" "watch"]}

{APIGroups:["rbac.authorization.k8s.io"], Resources:["serviceaccounts"], Verbs:["get" "list" "create" "delete" "patch" "update" "watch"]}

{APIGroups:["rbac.authorization.k8s.io"], Resources:["services"], Verbs:["get" "list" "create" "delete" "patch" "update" "watch"]}

{APIGroups:["rbac.authorization.k8s.io"], Resources:["services/finalizers"], Verbs:["get" "list" "create" "delete" "patch" "update" "watch"]}

{APIGroups:["route.openshift.io"], Resources:["routes"], Verbs:["get" "list" "create" "delete" "patch" "update" "watch"]}

{APIGroups:["route.openshift.io"], Resources:["routes/custom-host"], Verbs:["get" "list" "create" "delete" "patch" "update" "watch"]}

Error from server (Forbidden): error when creating "STDIN": roles.rbac.authorization.k8s.io "awx-operator-leader-election-role" is forbidden: user "u-w4453qrfpa" (groups=["azuread_group:// "system:authenticated" "system:cattle:authenticated"]) is attempting to grant RBAC permissions not currently held:

{APIGroups:["coordination.k8s.io"], Resources:["leases"], Verbs:["get" "list" "watch" "create" "update" "patch" "delete"]}

Error from server (Forbidden): error when retrieving current configuration of:

Resource: "rbac.authorization.k8s.io/v1, Resource=clusterroles", GroupVersionKind: "rbac.authorization.k8s.io/v1, Kind=ClusterRole"

Name: "awx-operator-metrics-reader", Namespace: ""

from server for: "STDIN": clusterroles.rbac.authorization.k8s.io "awx-operator-metrics-reader" is forbidden: User "u-w4453qrfpa" cannot get resource "clusterroles" in API group "rbac.authorization.k8s.io" at the cluster scope

Error from server (Forbidden): error when retrieving current configuration of:

Resource: "rbac.authorization.k8s.io/v1, Resource=clusterroles", GroupVersionKind: "rbac.authorization.k8s.io/v1, Kind=ClusterRole"

Name: "awx-operator-proxy-role", Namespace: ""

from server for: "STDIN": clusterroles.rbac.authorization.k8s.io "awx-operator-proxy-role" is forbidden: User "u-w4453qrfpa" cannot get resource "clusterroles" in API group "rbac.authorization.k8s.io" at the cluster scope

Error from server (NotFound): error when creating "STDIN": roles.rbac.authorization.k8s.io "awx-operator-awx-manager-role" not found

Error from server (NotFound): error when creating "STDIN": roles.rbac.authorization.k8s.io "awx-operator-leader-election-role" not found

Error from server (Forbidden): error when retrieving current configuration of:

Resource: "rbac.authorization.k8s.io/v1, Resource=clusterrolebindings", GroupVersionKind: "rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding"

Name: "awx-operator-proxy-rolebinding", Namespace: ""

from server for: "STDIN": clusterrolebindings.rbac.authorization.k8s.io "awx-operator-proxy-rolebinding" is forbidden: User "u-w4453qrfpa" cannot get resource "clusterrolebindings" in API group "rbac.authorization.k8s.io" at the cluster scope

make: *** [deploy] Error 1

I guess I'm confused about this PR I mentioned above and the fact it alludes to moving away from anything cluster scoped, yet I'm still seeing errors that indicate you can't really completely move everything from being cluster scoped?

Just can't seem to get past these errors to install via Operator and wondering what am I missing understanding about this.

Thanks, Dane