confirm. delivery regress: one extra queue

[Дмитрий Николаевич]

Hello every body

-------------------------------------------------------------------
OS: Debian GNU/Linux 12
Kernel: 6.1.0-37-amd64

-------------------------------------------------------------------
CPU info:

Architecture:                x86_64
  CPU op-mode(s):            32-bit, 64-bit
  Address sizes:             42 bits physical, 48 bits virtual
  Byte Order:                Little Endian
CPU(s):                      80
  On-line CPU(s) list:       0-79
Vendor ID:                   GenuineIntel
  Model name:                Intel(R) Xeon(R) Silver 4316 CPU @ 2.30GHz
    CPU family:              6
    Model:                   85
    Thread(s) per core:      1
    Core(s) per socket:      20
    Socket(s):               4
    Stepping:                7
    BogoMIPS:                4589.21
    Flags:                   fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon nopl xtopology tsc_re
                             liable nonstop_tsc cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dno
                             wprefetch invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced fsgsbase tsc_adjust bmi1 avx2 smep bmi2 invpcid rdseed adx smap xsaveopt arat md_clear flush_l1d arch_capabilities
Virtualization features:    
  Hypervisor vendor:         VMware
  Virtualization type:       full

----------------------------------------------------------------------
RabbitMQ: 3.12.10
Erlang: 25.3.2.7
Exchange type: direct
Queue type : classic
Manuak ack is used
Confirmation delivery is used

We have several daemons and use rabbitmq to pass data.
Rabbitmq is in docker.
All daemons are allocated on one host.
Most of them use plain type communication, and one of them is connected via SSL, and this service is wrapped in docker.
There are no problems with IO or cpu resources.

One daemon uses sender-selected distribution to pass same data to two different queues.
The first queue has consumers over plain type, where each client has own physical connection.
The second one has 8 consumers over ssl, and all clients share same physical connection using different channels.  

Some details for these consumers properties:

-----------------------------------------------------------------------------------------
Protocol Version tlsv1.2
Key Exchange Algorithm ecdhe_rsa
Cipher Algorithm aes_256_gcm
Hash Algorithm aead
Client properties
capabilities
authentication_failure_close: true
basic.nack: true
connection.blocked: true
consumer_cancel_notify: true
exchange_exchange_bindings: true
publisher_confirms: true
connection_name undefined
platform .NET
product RabbitMQ
version 6.4.0+ec8e55228ac781c816ef4bd666af1bbde72d8242


I notice big confirmation latency, but if the second queue is removed from recipients the latency gets smaller significant.
I know active consumers can impact on confirmation latency, is one physical connection over SSL bottleneck ?

Collected metrics are attached, round circle is a moment of removing the second queue. 

 

[Дмитрий Николаевич]

Some detaiils:
```rabbitmqctl list_queues pid name``` outs SecondQueueName pid as rabbitmq.1754947823.736.0

And observer util shows it in the top of reductions if i'm not mistaken:

```
Home(H)|Network(N)|System(S)|Ets(E)/Mnesia(M)|App(A)|Doc(D)|Plugin(P)recon:proc_count(memory, 37) Interval:5000ms     | 13Days 20:12:32  |
|Erlang/OTP 25 [erts-13.2.2.4] [source] [64-bit] [smp:80:80] [ds:80:80:10] [async-threads:1] [jit]                                        |  
|System     | Count/Limit           | System                    | Status                | Stat Info            | Size                     |
|Proc Count | 3698/1048576          | Version                   | 25                    | Active Task          | 1                        |  
|Port Count | 198/65536             | ps -o pcpu                | 121%                  | Context Switch       | 18402578426              |
|Atom Count | 48013/5000000         | ps -o pmem                | 2.1%                  | Reds(Total/SinceLast | 7939287018997/38770119   |
|Mem Type   | Size                  | Mem Type                  | Size                  | IO/GC:(5000ms)       | Total/Increments         |
|Total      | 3.0131 GB    | 100.0% | Binary                    | 255.9303 MB  | 08.29% | IO Output            | 76532.5933 GB/334.7653 M |  
|Process    | 2.6440 GB    | 87.75% | Code                      | 36.9494 MB   | 01.20% | IO Input             | 45575.3486 GB/291.8460 M |  
|Atom       | 1.3746 MB    | 00.04% | Port Parallelism (+spp)   | false                 | Gc Count             | 2915940940/11588         |  
|Ets        | 24.3087 MB   | 00.79% | RunQueue                  | 0                     | Gc Words Reclaimed   | 6327197053404/21734289   |
|No | Pid        |     Memory   |Name or Initial Call                  |           Reductions| MsgQueue |Current Function                 |
|1  |<0.736.0>   |    2.5460 GB |rabbit_prequeue:init/1                |        2448208909655| 0        |gen_server2:process_next_msg/1   |
|2  |<0.760.0>   |    3.9585 MB |rabbit_prequeue:init/1                |          23887063188| 0        |gen_server2:process_next_msg/1   |
|3  |<0.781.0>   |    1.7233 MB |rabbit_prequeue:init/1                |         140696876981| 0        |gen_server2:process_next_msg/1   |
```

понедельник, 25 августа 2025 г. в 13:53:38 UTC+3, Дмитрий Николаевич:

[Michal Kuratczyk]

RabbitMQ 3.12 reached end-of-life 18 month ago. We are not going to investigate problems in that version.

There's a good chance your problem will just go away if you upgrade - you probably use classic queues v1,

while classic queues v2 are the only version available in RabbitMQ 4.x (the migration will happen automatically

if you upgrade, but you can also do it earlier). v2 is a major rewrite of the internals and significantly

improves performance in many situations.

If you use classic queue mirroring, you should migrate to quorum queues before you upgrade to 4.x.

Best,

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/rabbitmq-users/df11569d-dbb6-4399-be56-e69abe2ac61bn%40googlegroups.com.

--

Michal

RabbitMQ Team

[Дмитрий Николаевич]

Ok, tnanks

вторник, 26 августа 2025 г. в 11:09:55 UTC+3, Michal Kuratczyk:

[Дмитрий Николаевич]

purge of second queue helped. it's strange, but allocated memory was almost 3G for emtpy queue

вторник, 26 августа 2025 г. в 11:09:55 UTC+3, Michal Kuratczyk:

RabbitMQ 3.12 reached end-of-life 18 month ago. We are not going to investigate problems in that version.