RabbitMQ not startiing up with changes in Kubernetes Security Context

[Aniket Patil]

Hi, 

I am deploying rabbitMQ in kubernetes using stateful set and persistent volumes. 

In my intial configuration I have the following in my secuirty context of the kubernetes deployment 

      securityContext:
        supplementalGroups:
        runAsUser: 100
        runAsGroup: 101
        fsGroup : 101 

Everything works fine I can cluster and use the RabbitMQ. 


Due to request in recent changes I am requested to change the Secuirty context   with arbitrary user ids all other things remain same. 

eg. 
      securityContext:
        supplementalGroups:
        - 117932853
        fsGroup: 117932853

This needs to applied to the existing deployment.  After the update my RabbitMQ pods fail to start with the following error below. 

Please not my volume is mounted on path /var/lib/rabbitmq 


I know its permissions issue,  Can I get some pointers to fix this issue.  Any help is appreciated. 

Thanks,
Aniket





Error log from the POD.


2022-06-29 00:05:31.054088+00:00 [warn] <0.130.0> Failed to write PID file "/var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local.pid": permission denied
2022-06-29 00:05:32.555239+00:00 [info] <0.229.0> Feature flags: list of feature flags found:
2022-06-29 00:05:32.555326+00:00 [info] <0.229.0> Feature flags:   [x] implicit_default_bindings
2022-06-29 00:05:32.555365+00:00 [info] <0.229.0> Feature flags:   [x] maintenance_mode_status
2022-06-29 00:05:32.555384+00:00 [info] <0.229.0> Feature flags:   [x] quorum_queue
2022-06-29 00:05:32.555467+00:00 [info] <0.229.0> Feature flags:   [x] stream_queue
2022-06-29 00:05:32.555485+00:00 [info] <0.229.0> Feature flags:   [x] user_limits
2022-06-29 00:05:32.555504+00:00 [info] <0.229.0> Feature flags:   [x] virtual_host_metadata
2022-06-29 00:05:32.555581+00:00 [info] <0.229.0> Feature flags: feature flag states written to disk: yes
2022-06-29 00:05:32.976618+00:00 [noti] <0.44.0> Application syslog exited with reason: stopped
2022-06-29 00:05:32.976712+00:00 [noti] <0.229.0> Logging: switching to configured handler(s); following messages may not be visible in this log output
2022-06-29 00:05:32.998017+00:00 [noti] <0.229.0> Logging: configured log handlers are now ACTIVE

BOOT FAILED
===========
Error during startup: {error,
                          {could_not_write_file,
                              "/var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/cluster_nodes.config",
                              eacces}}

2022-06-29 00:05:33.046478+00:00 [erro] <0.229.0>
2022-06-29 00:05:33.046478+00:00 [erro] <0.229.0> BOOT FAILED
2022-06-29 00:05:33.046478+00:00 [erro] <0.229.0> ===========
2022-06-29 00:05:33.046478+00:00 [erro] <0.229.0> Error during startup: {error,
2022-06-29 00:05:33.046478+00:00 [erro] <0.229.0>                           {could_not_write_file,
2022-06-29 00:05:33.046478+00:00 [erro] <0.229.0>                               "/var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/cluster_nodes.config",
2022-06-29 00:05:33.046478+00:00 [erro] <0.229.0>                               eacces}}
2022-06-29 00:05:33.046478+00:00 [erro] <0.229.0>
2022-06-29 00:05:34.047356+00:00 [erro] <0.228.0>   crasher:
2022-06-29 00:05:34.047356+00:00 [erro] <0.228.0>     initial call: application_master:init/4
2022-06-29 00:05:34.047356+00:00 [erro] <0.228.0>     pid: <0.228.0>
2022-06-29 00:05:34.047356+00:00 [erro] <0.228.0>     registered_name: []
2022-06-29 00:05:34.047356+00:00 [erro] <0.228.0>     exception exit: {{could_not_write_file,
2022-06-29 00:05:34.047356+00:00 [erro] <0.228.0>                          "/var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/cluster_nodes.config",
2022-06-29 00:05:34.047356+00:00 [erro] <0.228.0>                          eacces},
2022-06-29 00:05:34.047356+00:00 [erro] <0.228.0>                      {rabbit,start,[normal,[]]}}
2022-06-29 00:05:34.047356+00:00 [erro] <0.228.0>       in function  application_master:init/4 (application_master.erl, line 142)
2022-06-29 00:05:34.047356+00:00 [erro] <0.228.0>     ancestors: [<0.227.0>]
2022-06-29 00:05:34.047356+00:00 [erro] <0.228.0>     message_queue_len: 1
2022-06-29 00:05:34.047356+00:00 [erro] <0.228.0>     messages: [{'EXIT',<0.229.0>,normal}]
2022-06-29 00:05:34.047356+00:00 [erro] <0.228.0>     links: [<0.227.0>,<0.44.0>]
2022-06-29 00:05:34.047356+00:00 [erro] <0.228.0>     dictionary: []
2022-06-29 00:05:34.047356+00:00 [erro] <0.228.0>     trap_exit: true
2022-06-29 00:05:34.047356+00:00 [erro] <0.228.0>     status: running
2022-06-29 00:05:34.047356+00:00 [erro] <0.228.0>     heap_size: 1598
2022-06-29 00:05:34.047356+00:00 [erro] <0.228.0>     stack_size: 28
2022-06-29 00:05:34.047356+00:00 [erro] <0.228.0>     reductions: 201
2022-06-29 00:05:34.047356+00:00 [erro] <0.228.0>   neighbours:
2022-06-29 00:05:34.047356+00:00 [erro] <0.228.0>
2022-06-29 00:05:34.056384+00:00 [noti] <0.44.0> Application rabbit exited with reason: {{could_not_write_file,"/var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/cluster_nodes.config",eacces},{rabbit,start,[normal,[]]}}
{"Kernel pid terminated",application_controller,"{application_start_failure,rabbit,{{could_not_write_file,\"/var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/cluster_nodes.config\",eacces},{rabbit,start,[normal,[]]}}}"}
Kernel pid terminated (application_controller) ({application_start_failure,rabbit,{{could_not_write_file,"/var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/cluster_nodes.config",eacces},{rabbit,start,[normal,[]]}}})

Crash dump is being written to: /var/log/rabbitmq/erl_crash.dump...done

[Michal Kuratczyk]

Hi,

I guess an initContainer that changes the permissions should do the trick.

Best,

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/a881bad7-2c44-4b6a-8b65-3004a08882e7n%40googlegroups.com.

--

Michał

RabbitMQ team

[Aniket Patil]

Thanks Micheal. 

I do have an init container as part of my deployment. I added the following in my initcontainer 

chown -R 117932853:117932853  /var/lib/rabbitmq  (  /var/lib/rabbitmq is the mounted PV   117932853 is user and group ids). 

When i run the deployment  get the following error.   Am i missing something. Kindly advise. 

Thanks,
Aniket


chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/cluster_nodes.config: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/rabbit_durable_route.DCD: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/rabbit_topic_permission.DCD: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/rabbit_runtime_parameters.DCD: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/rabbit_user_permission.DCD: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/rabbit_durable_queue.DCD: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/rabbit_user.DCL: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/.config: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/.vhost: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent/0.rdq: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent: Operation not permitted      
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent: Operation not permitted      
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/recovery.dets: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient/0.rdq: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient: Operation not permitted      
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient: Operation not permitted      
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/msg_stores/vhosts: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/msg_stores/vhosts: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/msg_stores: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/msg_stores: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/nodes_running_at_shutdown: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/schema.DAT: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/quorum/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/meta.dets: Operation not permitt
ed
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/quorum/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/00000001.wal: Operation not perm
itted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/quorum/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/names.dets: Operation not permit
ted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/quorum/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local: Operation not permitted        
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/quorum/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local: Operation not permitted        
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/quorum: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/quorum: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/rabbit_durable_queue.DCL: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/coordination/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/meta.dets: Operation not p
ermitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/coordination/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/00000001.wal: Operation no
t permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/coordination/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/names.dets: Operation not
permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/coordination/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local: Operation not permitted  
bash-5.1$ chown -R 117932853:117932853 /var/lib/rabbitmq
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/cluster_nodes.config: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/rabbit_durable_route.DCD: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/rabbit_topic_permission.DCD: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/rabbit_runtime_parameters.DCD: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/rabbit_user_permission.DCD: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/rabbit_durable_queue.DCD: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/rabbit_user.DCL: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/.config: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/.vhost: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent/0.rdq: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent: Operation not permitted      
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent: Operation not permitted      
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/recovery.dets: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient/0.rdq: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient: Operation not permitted      
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient: Operation not permitted      
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/msg_stores/vhosts: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/msg_stores/vhosts: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/msg_stores: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/msg_stores: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/nodes_running_at_shutdown: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/schema.DAT: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/quorum/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/meta.dets: Operation not permitt
ed
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/quorum/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/00000001.wal: Operation not perm
itted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/quorum/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/names.dets: Operation not permit
ted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/quorum/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local: Operation not permitted        
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/quorum/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local: Operation not permitted        
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/quorum: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/quorum: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/rabbit_durable_queue.DCL: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/coordination/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/meta.dets: Operation not p
ermitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/coordination/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/00000001.wal: Operation no
t permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/coordination/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/names.dets: Operation not
permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/coordination/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local: Operation not permitted  
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/coordination/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local: Operation not permitted  
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/coordination: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/coordination: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/rabbit_vhost.DCD: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/rabbit_durable_exchange.DCD: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/LATEST.LOG: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/DECISION_TAB.LOG: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/rabbit_user.DCD: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/rabbit_vhost.DCL: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/rabbit_serial: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local/schema_version: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local.pid: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local-feature_flags: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local-plugins-expand: Operation not permitted
chown: /var/lib/rabbitmq/mnesia/rab...@arcgis-queue-store-0.arcgis-queue-store-internal.arcgis.svc.cluster.local-plugins-expand: Operation not permitted
chown: /var/lib/rabbitmq/mnesia: Operation not permitted