Re: [rabbitmq-users] Issues with switching to using RabbitMQ operator

241 views
Skip to first unread message
Message has been deleted

Michal Kuratczyk

unread,
Mar 21, 2024, 3:26:40 AM3/21/24
to rabbitm...@googlegroups.com
I don't understand why the statefulset's name is important here. You can declare a service that will expose the new cluster with the same name as before.

On Wed, Mar 20, 2024 at 10:41 AM Davit Kocharyan <d.koc...@sxope.com> wrote:
I have RabbitMQ cluster deployed by helm chart using terraform.
I want to switch to RabbitMQ operator and deploy RabbitmqCluster to replace existing one and I want to do that harmless. 
RabbitmqCluster automatically adds -server prefix to the statefulSet name, so I can't give it the same name as my existing cluster, how can I change it? 
Or is there a correct way to do that switch, so new one will take an existing pod and it's queues?

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/742b2109-2a41-4e3d-b5dd-c3e9224216f2n%40googlegroups.com.


--
Michal
RabbitMQ Team

This electronic communication and the information and any files transmitted with it, or attached to it, are confidential and are intended solely for the use of the individual or entity to whom it is addressed and may contain information that is confidential, legally privileged, protected by privacy laws, or otherwise restricted from disclosure to anyone else. If you are not the intended recipient or the person responsible for delivering the e-mail to the intended recipient, you are hereby notified that any use, copying, distributing, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited. If you received this e-mail in error, please return the e-mail to the sender, delete it from your computer, and destroy any printed copy of it.
Message has been deleted

Michal Kuratczyk

unread,
Mar 21, 2024, 5:48:05 AM3/21/24
to rabbitm...@googlegroups.com
Do you mean you want the operator-deployed instance to take-over/inherit the statefulset and the pods?
That's not supported and likely impossible.

On Thu, Mar 21, 2024 at 10:45 AM Davit Kocharyan <d.koc...@sxope.com> wrote:
Hey Michal, thanks for the answer, my current configuration in Kubernetes is like this
Screenshot 2024-03-21 132107.png

I am trying to add RabbitmqCluster with the below config, how can I change it so it will take the existing "rabbitmq-standalone-0" pod and its queues?

resource "kubernetes_manifest" "kubernetes_manifest_rabbitmq_standalone" {
for_each = var.rabbitmq_operator_enabled && var.rabbitmq_standalone_enabled ? { enabled = true } : {}
manifest = {
apiVersion = "rabbitmq.com/v1beta1"
kind = "RabbitmqCluster"
metadata = {
name = "rabbitmq-standalone"
namespace = "rabbitmq"
}
spec = {
replicas = 1
override = {
statefulSet = {
spec = {
template = {
metadata = {
name = "rabbitmq-standalone"
}
}
}
}
}
service = {
type = "ClusterIP"
}
resources = {
requests = {
memory = var.rabbitmq_standalone_resources_request_memory
cpu = var.rabbitmq_standalone_resources_request_cpu
}
limits = {
memory = var.rabbitmq_standalone_resources_limit_memory
cpu = var.rabbitmq_standalone_resources_limit_cpu
}
}
persistence = {
storageClassName = var.rabbitmq_storage_class
storage = var.rabbitmq_standalone_disk_size
}
rabbitmq = {
additionalPlugins = [
"rabbitmq_management",
"rabbitmq_peer_discovery_k8s"
]
additionalConfig = join("\n", [
"default_vhost = physician-portal",
"log.file.level = error",
"log.console.level = error",
"consumer_timeout = ${var.rabbitmq_consumer_timeout_millisec}",
"default_user = ${local.rabbiqm_username}",
"default_pass = ${local.rabbiqm_password}",
])
}
}
}
}Screenshot 2024-03-21 134319.png
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/7ded6a18-fb0d-425b-bab1-8c3025e0cdc1n%40googlegroups.com.
Message has been deleted

Vilius Šumskas

unread,
Mar 21, 2024, 12:13:16 PM3/21/24
to rabbitm...@googlegroups.com

The operator itself doesn‘t do anything special regarding graceful shutdown of the RabbitMQ pods. All it does is it sets configuration for the RabbitMQ instances it controls during the creation and updates of the cluster. Given that configuration not only includes graceful shutdown preStop hook but all sorts of other options, which could be completely different from your current configuration, I’m afraid it won’t be feasible for you to try to achieve such migration.

 

You either have to recreate your clusters using operator and redirect applications to use them, or just add needed configuration manually to your current clusters, if you cannot do the former.

 

Though I won’t recommend it for every use case, we actually did the latter because we were in the similar boat as you https://groups.google.com/g/rabbitmq-users/c/YgP9aTBaRVs

 

--

    Vilius

 

From: rabbitm...@googlegroups.com <rabbitm...@googlegroups.com> On Behalf Of Davit Kocharyan
Sent: Thursday, March 21, 2024 1:13 PM
To: rabbitm...@googlegroups.com
Subject: Re: [rabbitmq-users] Issues with switching to using RabbitMQ operator

 

Yes, we have about 60 clusters and it will be a huge work to change configs for all of them, the reason we wanted to switch to the operator is because it supports graceful shutdown.

 

On Thu, Mar 21, 2024 at 1:48 PM 'Michal Kuratczyk' via rabbitmq-users <rabbitm...@googlegroups.com> wrote:

Do you mean you want the operator-deployed instance to take-over/inherit the statefulset and the pods?

That's not supported and likely impossible.

 

On Thu, Mar 21, 2024 at 10:45 AM Davit Kocharyan <d.koc...@sxope.com> wrote:

Hey Michal, thanks for the answer, my current configuration in Kubernetes is like this

}

On Thursday, March 21, 2024 at 11:26:40 AM UTC+4 Michal Kuratczyk wrote:

I don't understand why the statefulset's name is important here. You can declare a service that will expose the new cluster with the same name as before.

 

On Wed, Mar 20, 2024 at 10:41 AM Davit Kocharyan <d.koc...@sxope.com> wrote:

I have RabbitMQ cluster deployed by helm chart using terraform.

I want to switch to RabbitMQ operator and deploy RabbitmqCluster to replace existing one and I want to do that harmless. 

RabbitmqCluster automatically adds -server prefix to the statefulSet name, so I can't give it the same name as my existing cluster, how can I change it? 
Or is there a correct way to do that switch, so new one will take an existing pod and it's queues?

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/742b2109-2a41-4e3d-b5dd-c3e9224216f2n%40googlegroups.com.


 

--

Michal

RabbitMQ Team


This electronic communication and the information and any files transmitted with it, or attached to it, are confidential and are intended solely for the use of the individual or entity to whom it is addressed and may contain information that is confidential, legally privileged, protected by privacy laws, or otherwise restricted from disclosure to anyone else. If you are not the intended recipient or the person responsible for delivering the e-mail to the intended recipient, you are hereby notified that any use, copying, distributing, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited. If you received this e-mail in error, please return the e-mail to the sender, delete it from your computer, and destroy any printed copy of it.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/7ded6a18-fb0d-425b-bab1-8c3025e0cdc1n%40googlegroups.com.


 

--

Michal

RabbitMQ Team


This electronic communication and the information and any files transmitted with it, or attached to it, are confidential and are intended solely for the use of the individual or entity to whom it is addressed and may contain information that is confidential, legally privileged, protected by privacy laws, or otherwise restricted from disclosure to anyone else. If you are not the intended recipient or the person responsible for delivering the e-mail to the intended recipient, you are hereby notified that any use, copying, distributing, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited. If you received this e-mail in error, please return the e-mail to the sender, delete it from your computer, and destroy any printed copy of it.

--
You received this message because you are subscribed to a topic in the Google Groups "rabbitmq-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/rabbitmq-users/qhj7JkFw-hQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to rabbitmq-user...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/CALWErWLBvNusm%2BTX6LSPwWAmeiPUu-B3JYW-M%2BAT4WPv_Us_Bg%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.

Message has been deleted

Vilius Šumskas

unread,
Mar 28, 2024, 6:00:36 AM3/28/24
to rabbitm...@googlegroups.com

Yes, as mentioned in my previous email, you just need to adjust preStop hook and decide on the proper value of terminationGracePeriodSeconds in your existing environments. The details are in the thread link.

 

One caveat, after applying these settings cluster pods will still have to be restarted. This restart, of course, will use old shutdown configuration once.

 

--

    Vilius

 

From: rabbitm...@googlegroups.com <rabbitm...@googlegroups.com> On Behalf Of Davit Kocharyan
Sent: Thursday, March 28, 2024 9:16 AM
To: rabbitmq-users <rabbitm...@googlegroups.com>
Subject: Re: [rabbitmq-users] Issues with switching to using RabbitMQ operator

 

Thanks, Vilius for the answer, in that case, migration is not an option for us, is there any solution to implement a graceful restart in the existing environment?

On Thursday, March 21, 2024 at 8:13:16 PM UTC+4 Vilius Šumskas wrote:

The operator itself doesn‘t do anything special regarding graceful shutdown of the RabbitMQ pods. All it does is it sets configuration for the RabbitMQ instances it controls during the creation and updates of the cluster. Given that configuration not only includes graceful shutdown preStop hook but all sorts of other options, which could be completely different from your current configuration, I’m afraid it won’t be feasible for you to try to achieve such migration.

 

You either have to recreate your clusters using operator and redirect applications to use them, or just add needed configuration manually to your current clusters, if you cannot do the former.

 

Though I won’t recommend it for every use case, we actually did the latter because we were in the similar boat as you https://groups.google.com/g/rabbitmq-users/c/YgP9aTBaRVs

 

--

    Vilius

 

From: rabbitm...@googlegroups.com <rabbitm...@googlegroups.com> On Behalf Of Davit Kocharyan
Sent: Thursday, March 21, 2024 1:13 PM
To: rabbitm...@googlegroups.com
Subject: Re: [rabbitmq-users] Issues with switching to using RabbitMQ operator

 

Yes, we have about 60 clusters and it will be a huge work to change configs for all of them, the reason we wanted to switch to the operator is because it supports graceful shutdown.

 

On Thu, Mar 21, 2024 at 1:48 PM 'Michal Kuratczyk' via rabbitmq-users <rabbitm...@googlegroups.com> wrote:

Do you mean you want the operator-deployed instance to take-over/inherit the statefulset and the pods?

That's not supported and likely impossible.

 

On Thu, Mar 21, 2024 at 10:45 AM Davit Kocharyan <d.koc...@sxope.com> wrote:

Hey Michal, thanks for the answer, my current configuration in Kubernetes is like this

Image removed by sender.

}Image removed by sender.

Message has been deleted

Vilius Šumskas

unread,
Apr 8, 2024, 3:07:26 PM4/8/24
to rabbitm...@googlegroups.com

Hi,

 

this doesn‘t look correct as Kubernetes will glue this into „rabbitmq-upgrade await_online_quorum_plus_one rabbitmq-upgrade await_online_synchronized_mirror rabbitmq-upgrade drain“. You are also missing timeout options which will mean rabbitmq-upgrade will wait forever. What is OK and what is not depends on your environment.

 

For us it is:

 

          preStop:

            exec:

              command: [“- /bin/sh”, “-c”, “rabbitmq-upgrade await_online_synchronized_mirror -t 3600 && rabbitmq-upgrade drain -t 3600”]

 

But that’s just because we don’t use quorum queues at the moment. The timeout value is also adjusted to our support/incident processes. Don’t forget to also adjust terminationGracePeriodSeconds pod setting.

 

Or you can just simply copy the value RabbitMQ Cluster Operator provides by deploying fresh operator into testing environment.

 

Anyway, you should get to know how Kubernetes hooks and shell execution works in general, before doing anything.

 

--

    Vilius

 

From: rabbitm...@googlegroups.com <rabbitm...@googlegroups.com> On Behalf Of Davit Kocharyan
Sent: Monday, April 8, 2024 6:47 PM
To: rabbitm...@googlegroups.com
Subject: Re: [rabbitmq-users] Issues with switching to using RabbitMQ operator

 

Thanks Vilius, 

Can you please confirm if this is the correct preStop hook to use?

command = ["rabbitmq-upgrade await_online_quorum_plus_one", "rabbitmq-upgrade await_online_synchronized_mirror", "rabbitmq-upgrade drain"]

I have seen people are using this as well 

command = ["rabbitmqctl", "stop_app"]

 

Best Regards

Davit

 

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.

Vilius Šumskas

unread,
Apr 8, 2024, 3:16:13 PM4/8/24
to rabbitm...@googlegroups.com

Sorry, the hook in my last email should have been: 

 

             command: [“/bin/sh”, “-c”, “rabbitmq-upgrade await_online_synchronized_mirror -t 3600 && rabbitmq-upgrade drain -t 3600”]

 

--

    Vilius

Reply all
Reply to author
Forward
0 new messages