pgupgrade object does not start

[Jesper Holmberg]

We're running PGO 5.3 on OpenShift. We have a cluster that we'd like to upgrade from postgres 14 to 15.

Initially we found that the pgupgrade CDR was not installed, so we manually installed the CDR. This seems to have gone well.

However, things do not work as expected when we create a pgupgrade object.

This is what we're installing:

apiVersion: postgres-operator.crunchydata.com/v1beta1
kind: PGUpgrade
metadata:
  name: sbr-upgrade
  namespace: bis-sbr-db
spec:
  image: registry.developers.crunchydata.com/crunchydata/crunchy-upgrade:ubi8-5.3.0-0
  postgresClusterName: sbr-dev
  fromPostgresVersion: 14
  toPostgresVersion: 15

The result is no error, but neither are we seeing any effect from this. When looking at the object, it never receives a status element. No upgrade pod i started. So the effect is as though the pgupgrade object was ignored by the operator, and nothing was initialized. No logs seem to contain any errors.

How can I go about solving this?

[drew.s...@crunchydata.com]

Hello,

In PGO 5.3, the upgrade reconciler needs to be deployed separately. (In newer versions, the operator will include reconcilers for the both Custom Resources.)

So, in order to deploy the upgrade manager separately, a few questions:

1) Did you get the upgrade CRD from the postgres-operator-examples repo here?

2) How did you deploy the PGO operator in the first place? OpenShift makes it very easy to deploy the operator (through OLM), but to deploy the upgrade, you'll need to use kubectl.

I recommend reading through the documentation here to get an understanding of using kubectl and kustomize to install components into your cluster. Then, I recommend going over to the postgres-operator-examples repo and looking in the kustomize/install directory. There you will see the various objects that are needed to install both the operator and the upgrade manager. You already have the operator running so you will only need to install the items that are related to the upgrade manager. If you have a good understanding of kustomize and what is happening there, you can make edits to those files and use kustomize to install the necessary files. If that proves daunting, you can also take the text block below, stick it in a yaml file (make sure it has proper yaml formatting (email isn't the best way to share this, unfortunately)), and then install that file using kubectl (this SHOULD have everything needed to use the upgrade manager although it does assume that you are setup to manage PostgreSQL cluster-wide (again, see the documentation above)):

 --- apiVersion: v1 kind: ServiceAccount metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"v1","kind":"ServiceAccount","metadata":{"annotations":{},"labels":{"app.kubernetes.io/name":"pgo","app.kubernetes.io/version":"5.3.1","postgres-operator.crunchydata.com/control-plane":"postgres-operator-upgrade"},"name":"postgres-operator-upgrade","namespace":"postgres-operator"}} labels: app.kubernetes.io/name: pgo app.kubernetes.io/version: 5.3.1 postgres-operator.crunchydata.com/control-plane: postgres-operator-upgrade name: postgres-operator-upgrade namespace: postgres-operator --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"rbac.authorization.k8s.io/v1","kind":"ClusterRole","metadata":{"annotations":{},"creationTimestamp":null,"labels":{"app.kubernetes.io/name":"pgo","app.kubernetes.io/version":"5.3.1","postgres-operator.crunchydata.com/control-plane":"postgres-operator-upgrade"},"name":"postgres-operator-upgrade"},"rules":[{"apiGroups":[""],"resources":["endpoints"],"verbs":["delete","get","list","watch"]},{"apiGroups":["apps"],"resources":["statefulsets"],"verbs":["list","watch"]},{"apiGroups":["batch"],"resources":["jobs"],"verbs":["create","delete","list","patch","watch"]},{"apiGroups":["postgres-operator.crunchydata.com"],"resources":["pgupgrades"],"verbs":["get","list","watch"]},{"apiGroups":["postgres-operator.crunchydata.com"],"resources":["pgupgrades/finalizers"],"verbs":["patch","update"]},{"apiGroups":["postgres-operator.crunchydata.com"],"resources":["pgupgrades/status"],"verbs":["get","patch"]},{"apiGroups":["postgres-operator.crunchydata.com"],"resources":["postgresclusters"],"verbs":["get","list","watch"]},{"apiGroups":["postgres-operator.crunchydata.com"],"resources":["postgresclusters/status"],"verbs":["patch"]}]} creationTimestamp: null labels: app.kubernetes.io/name: pgo app.kubernetes.io/version: 5.3.1 postgres-operator.crunchydata.com/control-plane: postgres-operator-upgrade name: postgres-operator-upgrade rules: - apiGroups: - "" resources: - endpoints verbs: - delete - get - list - watch - apiGroups: - apps resources: - statefulsets verbs: - list - watch - apiGroups: - batch resources: - jobs verbs: - create - delete - list - patch - watch - apiGroups: - postgres-operator.crunchydata.com resources: - pgupgrades verbs: - get - list - watch - apiGroups: - postgres-operator.crunchydata.com resources: - pgupgrades/finalizers verbs: - patch - update - apiGroups: - postgres-operator.crunchydata.com resources: - pgupgrades/status verbs: - get - patch - apiGroups: - postgres-operator.crunchydata.com resources: - postgresclusters verbs: - get - list - watch - apiGroups: - postgres-operator.crunchydata.com resources: - postgresclusters/status verbs: - patch --- apiVersion: apps/v1 kind: Deployment metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"apps/v1","kind":"Deployment","metadata":{"annotations":{},"labels":{"app.kubernetes.io/name":"pgo","app.kubernetes.io/version":"5.3.1","postgres-operator.crunchydata.com/control-plane":"postgres-operator-upgrade"},"name":"pgo-upgrade","namespace":"postgres-operator"},"spec":{"replicas":1,"selector":{"matchLabels":{"postgres-operator.crunchydata.com/control-plane":"postgres-operator-upgrade"}},"strategy":{"type":"Recreate"},"template":{"metadata":{"labels":{"app.kubernetes.io/name":"pgo","app.kubernetes.io/version":"5.3.1","postgres-operator.crunchydata.com/control-plane":"postgres-operator-upgrade"}},"spec":{"containers":[{"env":[{"name":"PGO_NAMESPACE","valueFrom":{"fieldRef":{"fieldPath":"metadata.namespace"}}},{"name":"CRUNCHY_DEBUG","value":"true"},{"name":"RELATED_IMAGE_PGUPGRADE","value":"registry.developers.crunchydata.com/crunchydata/crunchy-upgrade:ubi8-5.3.1-0"}],"image":"registry.developers.crunchydata.com/crunchydata/postgres-operator-upgrade:ubi8-5.3.1-0","name":"operator","securityContext":{"allowPrivilegeEscalation":false,"capabilities":{"drop":["ALL"]},"readOnlyRootFilesystem":true,"runAsNonRoot":true}}],"serviceAccountName":"postgres-operator-upgrade"}}}} labels: app.kubernetes.io/name: pgo app.kubernetes.io/version: 5.3.1 postgres-operator.crunchydata.com/control-plane: postgres-operator-upgrade name: pgo-upgrade namespace: postgres-operator spec: replicas: 1 selector: matchLabels: postgres-operator.crunchydata.com/control-plane: postgres-operator-upgrade strategy: type: Recreate template: metadata: labels: app.kubernetes.io/name: pgo app.kubernetes.io/version: 5.3.1 postgres-operator.crunchydata.com/control-plane: postgres-operator-upgrade spec: containers: - env: - name: PGO_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace - name: CRUNCHY_DEBUG value: "true" - name: RELATED_IMAGE_PGUPGRADE value: registry.developers.crunchydata.com/crunchydata/crunchy-upgrade:ubi8-5.3.1-0 image: registry.developers.crunchydata.com/crunchydata/postgres-operator-upgrade:ubi8-5.3.1-0 name: operator securityContext: allowPrivilegeEscalation: false capabilities: drop: - ALL readOnlyRootFilesystem: true runAsNonRoot: true serviceAccountName: postgres-operator-upgrade ---

Hope this helps!

Regards,

Drew