Cluster Operator: Policy that mirrors queues to 2 nodes

[Ralph Otowo]

We have an old installation of RabbitMQ using the now deprecated diy-kubernetes setup guide and are looking to migrate to the new and improved cluster operator. I'm currently stuck trying to replicate the policy below into a kubernetes policy manifest file.

This is the policy file we have that works:

apiVersion: rabbitmq.com/v1beta1
kind: Policy
metadata:
  name: staging-ha
spec:
  name: ha
  vhost: "/"
  pattern: "^(?!amq\\.).*"
  applyTo: "queues"
  priority: 1
  definition:
    ha-mode: "all"
    ha-sync-mode: "automatic"
    # ha-params: 2
  rabbitmqClusterReference:
    name: rabbitmq-staging

The file above when applied results in the following policy:

I have tried changing the ha-mode to 'exactly' and un-commenting the ha-params line to get a policy that matches the one in the screenshot above but I've been unsuccessful with the following error from the messaging topology operator pod's logs:

Validation failed\n\nha-mode="exactly" takes an integer, none │
│ given\n","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/bitnami/blacksmith-sandox/rmq-messaging-topology-operator-1.8.0/pkg/mod/sigs.k8s. │
│ io/controlle...@v0.12.3/pkg/internal/controller/controller.go:273\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/bitnami/blacksmith-sandox/rmq-messagin │
│ g-topology-operator-1.8.0/pkg/mod/sigs.k8s.io/controlle...@v0.12.3/pkg/internal/controller/controller.go:234"}

I want to mirror my queues across two nodes, not all. Can anyone share steps to accomplish this using the cluster operator?

I can login to one of the pods and manually execute the following, but I'm looking to define it declaratively.

rabbitmqctl set_policy --priority 1 --apply-to queues ha '^(?!amq\.).*' '{"ha-mode":"exactly","ha-params":2,"ha-sync-mode":"automatic"}'

[Chunyi Lyu]

Hi Ralph,

Unfortunately I cannot reproduce the error. I was able to create a policy with the definition you've provided:

```

apiVersion: rabbitmq.com/v1beta1
kind: Policy
metadata:

  name: ha

spec:
  name: ha
  vhost: "/"
  pattern: "^(?!amq\\.).*"
  applyTo: "queues"
  priority: 1
  definition:

    ha-mode: "exactly"
    ha-sync-mode: "automatic"
    ha-params: 2
  rabbitmqClusterReference:
    name: sample

```

and the policy was created successfully and can be found in the server
```

rabbitmq@sample-server-0:/$ rabbitmqctl list_policies
Listing policies for vhost "/" ...
vhost    name    pattern    apply-to    definition    priority
/    ha    ^(?!amq\.).*    queues    {"ha-mode":"exactly","ha-params":2,"ha-sync-mode":"automatic"}    1
```

Could you please double check the indentation and format of your policy manifest, particularly section `spec.definition`? Messaging Topology Operator does not have any additional verification on the policy format. My best guess at the moment would be there is something wrong with how the policy definition is formatted and them it's parsed weirdly.

If the error persists, could you please let us know how did you install the operators and what's their versions?

Thanks,

Chunyi

[Ralph Otowo]

Hi Chunyi,

Thank you for your feedback. I've attached the policy file I'm struggling with. Maybe you will find something wrong with it, but it appears to follow the indentation and format of the policy manifest in your reply.

These are the steps I'm following to install the RabbitMQ Cluster Operator:
helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo update
helm template --include-crds -n rabbitmq rabbitmq bitnami/rabbitmq-cluster-operator > operator/base/template.yaml

I'm using the official RabbitMQ docker image from DockerHub. I have also attached the RabbitMQCluster manifest.

After re-installing the Cluster Operator and and applying the policy manifest, I'm getting a different error from messaging topology operator pod:
{"level":"error","ts":1669703178.3777633,"msg":"Reconciler error","controller":"policy","controllerGroup":"rabbitmq.com","controllerKind":"Policy","policy":{"name":"high-availability","namespace":"r │
│ abbitmq"},"namespace":"rabbitmq","name":"high-availability","reconcileID":"50561585-dc25-4500-93e9-d33a74534ed5","error":"Error: API responded with a 401 Unauthorized","stacktrace":"sigs.k8s.io/cont │
│ roller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/bitnami/blacksmith-sandox/rmq-messaging-topology-operator-1.8.0/pkg/mod/sigs.k8s.io/controlle...@v0.12.3/pkg/intern │
│ al/controller/controller.go:273\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/bitnami/blacksmith-sandox/rmq-messaging-topology-operator-1.8.0/pkg/mod/sigs.k │
│ 8s.io/controlle...@v0.12.3/pkg/internal/controller/controller.go:234"}

Best regards,

Ralph

[Ralph Otowo]

Hi Chunyi,

You're right, there seems to have been an issue with the policy manifest because it worked after clearing and re-indenting the relevant sections of the file. Thank you very much for the help.

The 401 unauthorized error I shared above was also reported as a reply to this thread, it happens whenever I create a cluster with a default_user and default_pass specified under the additionalConfig section of my RabbitMQCluster manifest. After commenting out the default credentials as shown in the attached manifest, I'm able to create RabbitMQ CRD objects without getting the 401 error.

I will create a separate discussion for the 401 error as the purpose of creating this thread has been resolved. Thank you!