[Errno 104] Connection reset by peer when running multiple docker containers & RabbitMQ/pika
372 views
Skip to first unread message
the_alchemist
Jan 10, 2020, 8:24:41 AM1/10/20
to Pika
Hi,
I have a number of dockers running in a docker-compose file together with a RabbitMQ container. I have been using pika to create a publish/subscribe type of communication.
I noticed that after a while of everything working as expected one of the containers crashes and throws the error below:
container_1 | ERROR:pika.adapters.utils.io_services_utils:_AsyncBaseTransport._produce() failed, aborting connection: error=error(104, 'Connection reset by peer'); sock=<socket._socketobject object at 0x7fad1f4c3210>; Caller's stack:
container_1 | Traceback (most recent call last):
container_1 | File "/usr/local/lib/python2.7/dist-packages/pika/adapters/utils/io_services_utils.py", line 1097, in _on_socket_writable
container_1 | self._produce()
container_1 | File "/usr/local/lib/python2.7/dist-packages/pika/adapters/utils/io_services_utils.py", line 820, in _produce
container_1 | self._tx_buffers[0])
container_1 | File "/usr/local/lib/python2.7/dist-packages/pika/adapters/utils/io_services_utils.py", line 79, in retry_sigint_wrap
container_1 | return func(*args, **kwargs)
container_1 | File "/usr/local/lib/python2.7/dist-packages/pika/adapters/utils/io_services_utils.py", line 861, in _sigint_safe_send
container_1 | return sock.send(data)
container_1 | error: [Errno 104] Connection reset by peer
container_1 | Traceback (most recent call last):
container_1 | File "/usr/local/lib/python2.7/dist-packages/pika/adapters/utils/io_services_utils.py", line 1097, in _on_socket_writable
container_1 | self._produce()
container_1 | File "/usr/local/lib/python2.7/dist-packages/pika/adapters/utils/io_services_utils.py", line 820, in _produce
container_1 | self._tx_buffers[0])
container_1 | File "/usr/local/lib/python2.7/dist-packages/pika/adapters/utils/io_services_utils.py", line 79, in retry_sigint_wrap
container_1 | return func(*args, **kwargs)
container_1 | File "/usr/local/lib/python2.7/dist-packages/pika/adapters/utils/io_services_utils.py", line 861, in _sigint_safe_send
container_1 | return sock.send(data)
container_1 | error: [Errno 104] Connection reset by peer
container_1 | ERROR:pika.adapters.base_connection:connection_lost: StreamLostError: ("Stream connection lost: error(104, 'Connection reset by peer')",)
container_1 | ERROR:pika.adapters.blocking_connection:Unexpected connection close detected: StreamLostError: ("Stream connection lost: error(104, 'Connection reset by peer')",)
rabbitmq_1 | 2020-01-10 10:44:30.471 [info] <0.11017.1> accepting AMQP connection <0.11017.1> (172.19.0.5:52234 -> 172.19.0.2:5672)
rabbitmq_1 | 2020-01-10 10:44:30.472 [info] <0.11017.1> connection <0.11017.1> (172.19.0.5:52234 -> 172.19.0.2:5672): user 'guest' authenticated and granted access to vhost '/'(Part of publisher code):
def publish(option, message):
# Connect to RabbitMQ on the localhost, if different machine use IP/hostname
connection = pika.BlockingConnection(pika.ConnectionParameters('rabbitmq', connection_attempts=5, retry_delay=5, heartbeat=300))
channel = connection.channel()
#'test' exchange to send the message to
channel.exchange_declare(exchange='test', exchange_type='direct')
channel.basic_publish(exchange='test', routing_key=option, body=message)
print(" [x] Sent %r:%r" % (option, message))
channel.close()
(Part of subscriber code):
def rabbitmqConnection(options, callback):
while True:
try:
connection = pika.BlockingConnection(pika.ConnectionParameters('rabbitmq', connection_attempts=5, retry_delay=5, heartbeat=300))
[...]
except Exception as e:
msg = ('ampq connection failed ({})'.format(str(e)))
[...]
channel = connection.channel()
channel.exchange_declare(exchange='test', exchange_type='direct')
result = channel.queue_declare(queue='', exclusive=True)
queue_name = result.method.queue
for scan_type in options:
channel.queue_bind(exchange='test', queue=queue_name, routing_key=scan_type)
print(" [*] Waiting for logs. To exit press CTRL+C")
# This specific callback should receive msg from hello queue
channel.basic_consume(queue=queue_name, on_message_callback=callback)
try:
# Never ending loop to wait for messages
channel.start_consuming()
except KeyboardInterrupt:
channel.stop_consuming()
except:
print('Connection closed. Reconnecting to RabbitMQ')
continue
channel.close()
break
Doing some investigation on the RabbitMQ container I see that every time the container crashes I am left with 1036 messages in a specific queue:
rabbitmqctl list_queues
Timeout: 60.0 seconds ...
Listing queues for vhost / ...
name messages
amq.gen-z02sxs2DAdTt4DEgGsKupQ 0
amq.gen-T0GevEXwRkFsXKXv3gLJjg 1
amq.gen-0Td8EsV2N6oMrhwqqCftxQ 1036
amq.gen-9hK9-3EQzxV0vbZD7UsHlA 1
amq.gen-Tu-AUp3kEa5L7PCQ2nyVgw 1
amq.gen-juFJWfXyen5RhCa_UGtydw 1
amq.gen-76GMHgToR33Ny1PkSsoFcg 1
amq.gen-exRRIRzjXoBSRieROE6w3A 0
Any idea on how to fix this?If you want any more info let me know.
Thanks a lot
Luke Bakken
Jan 10, 2020, 11:46:02 AM1/10/20
to Pika
Hello,
What is logged by RabbitMQ at the same time?
Hitesh Saini
Sep 10, 2024, 8:17:32 AM9/10/24
to Pika
Hi All,
Did we get any resolution on this as I am getting the same issue with same logs. I checked with RabbitMQ (a9s messaging service) and they are saying that everything is working fine at their end. Please help us to resolve this issue.
Did we get any resolution on this as I am getting the same issue with same logs. I checked with RabbitMQ (a9s messaging service) and they are saying that everything is working fine at their end. Please help us to resolve this issue.
Luke Bakken
Sep 10, 2024, 3:44:49 PM9/10/24
to Pika
Hello,
You're lucky I just happened to notice that you replied to a discussion that is over 4 years old.
One sentence is not nearly enough information to get support.
If you'd like support, start a discussion here:
Include the following information:
* Python version
* Pika version
* Logs / exceptions from your application
* Logs from RabbitMQ at the time of error
If RabbitMQ is a critical part of your infrastructure, you should consider a support contract - https://www.rabbitmq.com/contact#paid-support
Thank you -
Luke Bakken
Team RabbitMQ
Pika maintainer
Hitesh Saini
Sep 11, 2024, 8:15:06 AM9/11/24
to Pika
Hi Luke,
Please find the required details below:
Python version - 3.11.9
Pika version - 1.3.2
Pika version - 1.3.2
RabbitMQ logs - RabbitMQ Logs
https://drive.google.com/file/d/1tG7BBerQgdJHyWv6DNEILnKa69POiHFD/view?usp=drive_link
Our server - Application is deployed on DHC PaaS (Daimler hybrid cloud) and logs are below-
Our server - Application is deployed on DHC PaaS (Daimler hybrid cloud) and logs are below-
Log - 1
2024-09-05T18:02:29.035+05:30 [APP/PROC/WEB/0] [ERR] Traceback (most recent call last): 2024-09-05T18:02:29.036+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/app/data_loader/receiver.py", line 55, in consumer 2024-09-05T18:02:29.036+05:30 [APP/PROC/WEB/0] [ERR] channel.start_consuming() 2024-09-05T18:02:29.036+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 1883, in start_consuming 2024-09-05T18:02:29.036+05:30 [APP/PROC/WEB/0] [ERR] self._process_data_events(time_limit=None) 2024-09-05T18:02:29.036+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 2044, in _process_data_events 2024-09-05T18:02:29.036+05:30 [APP/PROC/WEB/0] [ERR] self.connection.process_data_events(time_limit=time_limit) 2024-09-05T18:02:29.036+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 842, in process_data_events 2024-09-05T18:02:29.036+05:30 [APP/PROC/WEB/0] [ERR] self._flush_output(common_terminator) 2024-09-05T18:02:29.036+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 523, in _flush_output 2024-09-05T18:02:29.036+05:30 [APP/PROC/WEB/0] [ERR] raise self._closed_result.value.error 2024-09-05T18:02:29.036+05:30 [APP/PROC/WEB/0] [ERR] pika.exceptions.StreamLostError: Stream connection lost: SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2427)') 2024-09-05T18:02:29.036+05:30 [APP/PROC/WEB/0] [OUT] None 2024-09-05T18:02:29.036+05:30 [APP/PROC/WEB/0] [ERR] Exception in thread Consumer: 2024-09-05T18:02:29.036+05:30 [APP/PROC/WEB/0] [ERR] Traceback (most recent call last): 2024-09-05T18:02:29.036+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/app/data_loader/receiver.py", line 55, in consumer 2024-09-05T18:02:29.037+05:30 [APP/PROC/WEB/0] [ERR] channel.start_consuming() 2024-09-05T18:02:29.037+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 1883, in start_consuming 2024-09-05T18:02:29.037+05:30 [APP/PROC/WEB/0] [ERR] self._process_data_events(time_limit=None) 2024-09-05T18:02:29.038+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 2044, in _process_data_events 2024-09-05T18:02:29.038+05:30 [APP/PROC/WEB/0] [ERR] self.connection.process_data_events(time_limit=time_limit) 2024-09-05T18:02:29.039+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 842, in process_data_events 2024-09-05T18:02:29.039+05:30 [APP/PROC/WEB/0] [ERR] self._flush_output(common_terminator) 2024-09-05T18:02:29.039+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 523, in _flush_output 2024-09-05T18:02:29.039+05:30 [APP/PROC/WEB/0] [ERR] raise self._closed_result.value.error 2024-09-05T18:02:29.040+05:30 [APP/PROC/WEB/0] [ERR] pika.exceptions.StreamLostError: Stream connection lost: SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2427)') 2024-09-05T18:02:29.040+05:30 [APP/PROC/WEB/0] [ERR] During handling of the above exception, another exception occurred: 2024-09-05T18:02:29.040+05:30 [APP/PROC/WEB/0] [ERR] Traceback (most recent call last): 2024-09-05T18:02:29.040+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/threading.py", line 1045, in _bootstrap_inner 2024-09-05T18:02:29.040+05:30 [APP/PROC/WEB/0] [ERR] self.run() 2024-09-05T18:02:29.040+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/threading.py", line 982, in run 2024-09-05T18:02:29.041+05:30 [APP/PROC/WEB/0] [ERR] self._target(*self._args, **self._kwargs) 2024-09-05T18:02:29.041+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/app/data_loader/receiver.py", line 59, in consumer 2024-09-05T18:02:29.042+05:30 [APP/PROC/WEB/0] [ERR] channel.stop_consuming() 2024-09-05T18:02:29.042+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 1896, in stop_consuming 2024-09-05T18:02:29.042+05:30 [APP/PROC/WEB/0] [ERR] self._cancel_all_consumers() 2024-09-05T18:02:29.043+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 1497, in _cancel_all_consumers 2024-09-05T18:02:29.043+05:30 [APP/PROC/WEB/0] [ERR] self.basic_cancel(consumer_tag) 2024-09-05T18:02:29.043+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 1774, in basic_cancel 2024-09-05T18:02:29.044+05:30 [APP/PROC/WEB/0] [ERR] assert (consumer_info.cancelled_by_broker or 2024-09-05T18:02:29.045+05:30 [APP/PROC/WEB/0] [ERR] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-09-05T18:02:29.045+05:30 [APP/PROC/WEB/0] [ERR] AssertionError: ctag1.755bfcde3bc54504bd2503040f954f8d
2024-09-05T18:02:29.035+05:30 [APP/PROC/WEB/0] [ERR] Traceback (most recent call last): 2024-09-05T18:02:29.036+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/app/data_loader/receiver.py", line 55, in consumer 2024-09-05T18:02:29.036+05:30 [APP/PROC/WEB/0] [ERR] channel.start_consuming() 2024-09-05T18:02:29.036+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 1883, in start_consuming 2024-09-05T18:02:29.036+05:30 [APP/PROC/WEB/0] [ERR] self._process_data_events(time_limit=None) 2024-09-05T18:02:29.036+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 2044, in _process_data_events 2024-09-05T18:02:29.036+05:30 [APP/PROC/WEB/0] [ERR] self.connection.process_data_events(time_limit=time_limit) 2024-09-05T18:02:29.036+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 842, in process_data_events 2024-09-05T18:02:29.036+05:30 [APP/PROC/WEB/0] [ERR] self._flush_output(common_terminator) 2024-09-05T18:02:29.036+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 523, in _flush_output 2024-09-05T18:02:29.036+05:30 [APP/PROC/WEB/0] [ERR] raise self._closed_result.value.error 2024-09-05T18:02:29.036+05:30 [APP/PROC/WEB/0] [ERR] pika.exceptions.StreamLostError: Stream connection lost: SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2427)') 2024-09-05T18:02:29.036+05:30 [APP/PROC/WEB/0] [OUT] None 2024-09-05T18:02:29.036+05:30 [APP/PROC/WEB/0] [ERR] Exception in thread Consumer: 2024-09-05T18:02:29.036+05:30 [APP/PROC/WEB/0] [ERR] Traceback (most recent call last): 2024-09-05T18:02:29.036+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/app/data_loader/receiver.py", line 55, in consumer 2024-09-05T18:02:29.037+05:30 [APP/PROC/WEB/0] [ERR] channel.start_consuming() 2024-09-05T18:02:29.037+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 1883, in start_consuming 2024-09-05T18:02:29.037+05:30 [APP/PROC/WEB/0] [ERR] self._process_data_events(time_limit=None) 2024-09-05T18:02:29.038+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 2044, in _process_data_events 2024-09-05T18:02:29.038+05:30 [APP/PROC/WEB/0] [ERR] self.connection.process_data_events(time_limit=time_limit) 2024-09-05T18:02:29.039+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 842, in process_data_events 2024-09-05T18:02:29.039+05:30 [APP/PROC/WEB/0] [ERR] self._flush_output(common_terminator) 2024-09-05T18:02:29.039+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 523, in _flush_output 2024-09-05T18:02:29.039+05:30 [APP/PROC/WEB/0] [ERR] raise self._closed_result.value.error 2024-09-05T18:02:29.040+05:30 [APP/PROC/WEB/0] [ERR] pika.exceptions.StreamLostError: Stream connection lost: SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2427)') 2024-09-05T18:02:29.040+05:30 [APP/PROC/WEB/0] [ERR] During handling of the above exception, another exception occurred: 2024-09-05T18:02:29.040+05:30 [APP/PROC/WEB/0] [ERR] Traceback (most recent call last): 2024-09-05T18:02:29.040+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/threading.py", line 1045, in _bootstrap_inner 2024-09-05T18:02:29.040+05:30 [APP/PROC/WEB/0] [ERR] self.run() 2024-09-05T18:02:29.040+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/threading.py", line 982, in run 2024-09-05T18:02:29.041+05:30 [APP/PROC/WEB/0] [ERR] self._target(*self._args, **self._kwargs) 2024-09-05T18:02:29.041+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/app/data_loader/receiver.py", line 59, in consumer 2024-09-05T18:02:29.042+05:30 [APP/PROC/WEB/0] [ERR] channel.stop_consuming() 2024-09-05T18:02:29.042+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 1896, in stop_consuming 2024-09-05T18:02:29.042+05:30 [APP/PROC/WEB/0] [ERR] self._cancel_all_consumers() 2024-09-05T18:02:29.043+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 1497, in _cancel_all_consumers 2024-09-05T18:02:29.043+05:30 [APP/PROC/WEB/0] [ERR] self.basic_cancel(consumer_tag) 2024-09-05T18:02:29.043+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 1774, in basic_cancel 2024-09-05T18:02:29.044+05:30 [APP/PROC/WEB/0] [ERR] assert (consumer_info.cancelled_by_broker or 2024-09-05T18:02:29.045+05:30 [APP/PROC/WEB/0] [ERR] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-09-05T18:02:29.045+05:30 [APP/PROC/WEB/0] [ERR] AssertionError: ctag1.755bfcde3bc54504bd2503040f954f8d
Log - 2
[ERR] ERROR:pika.adapters.utils.io_services_utils:_AsyncBaseTransport._produce() failed, aborting connection: error=SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2427)'); sock=<ssl.SSLSocket fd=13, family=2, type=1, proto=6, laddr=('172.29.86.124', 32772)>; Caller's stack:
2024-09-05T18:45:00.808+05:30 [APP/PROC/WEB/1] [ERR] Traceback (most recent call last):
2024-09-05T18:45:00.808+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/utils/io_services_utils.py", line 1308, in _produce
2024-09-05T18:45:00.808+05:30 [APP/PROC/WEB/1] [ERR] super(_AsyncSSLTransport, self)._produce()
2024-09-05T18:45:00.808+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/utils/io_services_utils.py", line 819, in _produce
2024-09-05T18:45:00.808+05:30 [APP/PROC/WEB/1] [ERR] num_bytes_sent = self._sigint_safe_send(self._sock,
2024-09-05T18:45:00.808+05:30 [APP/PROC/WEB/1] [ERR] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-09-05T18:45:00.808+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/utils/io_services_utils.py", line 79, in retry_sigint_wrap
2024-09-05T18:45:00.808+05:30 [APP/PROC/WEB/1] [ERR] return func(*args, **kwargs)
2024-09-05T18:45:00.808+05:30 [APP/PROC/WEB/1] [ERR] ^^^^^^^^^^^^^^^^^^^^^
2024-09-05T18:45:00.808+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/utils/io_services_utils.py", line 861, in _sigint_safe_send
2024-09-05T18:45:00.808+05:30 [APP/PROC/WEB/1] [ERR] return sock.send(data)
2024-09-05T18:45:00.808+05:30 [APP/PROC/WEB/1] [ERR] ^^^^^^^^^^^^^^^
2024-09-05T18:45:00.808+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/ssl.py", line 1242, in send
2024-09-05T18:45:00.808+05:30 [APP/PROC/WEB/1] [ERR] return self._sslobj.write(data)
2024-09-05T18:45:00.808+05:30 [APP/PROC/WEB/1] [ERR] ^^^^^^^^^^^^^^^^^^^^^^^^
2024-09-05T18:45:00.808+05:30 [APP/PROC/WEB/1] [ERR] ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:2427)
2024-09-05T18:45:00.808+05:30 [APP/PROC/WEB/1] [ERR] Traceback (most recent call last):
2024-09-05T18:45:00.808+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/utils/io_services_utils.py", line 1308, in _produce
2024-09-05T18:45:00.808+05:30 [APP/PROC/WEB/1] [ERR] super(_AsyncSSLTransport, self)._produce()
2024-09-05T18:45:00.808+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/utils/io_services_utils.py", line 819, in _produce
2024-09-05T18:45:00.808+05:30 [APP/PROC/WEB/1] [ERR] num_bytes_sent = self._sigint_safe_send(self._sock,
2024-09-05T18:45:00.808+05:30 [APP/PROC/WEB/1] [ERR] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-09-05T18:45:00.808+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/utils/io_services_utils.py", line 79, in retry_sigint_wrap
2024-09-05T18:45:00.808+05:30 [APP/PROC/WEB/1] [ERR] return func(*args, **kwargs)
2024-09-05T18:45:00.808+05:30 [APP/PROC/WEB/1] [ERR] ^^^^^^^^^^^^^^^^^^^^^
2024-09-05T18:45:00.808+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/utils/io_services_utils.py", line 861, in _sigint_safe_send
2024-09-05T18:45:00.808+05:30 [APP/PROC/WEB/1] [ERR] return sock.send(data)
2024-09-05T18:45:00.808+05:30 [APP/PROC/WEB/1] [ERR] ^^^^^^^^^^^^^^^
2024-09-05T18:45:00.808+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/ssl.py", line 1242, in send
2024-09-05T18:45:00.808+05:30 [APP/PROC/WEB/1] [ERR] return self._sslobj.write(data)
2024-09-05T18:45:00.808+05:30 [APP/PROC/WEB/1] [ERR] ^^^^^^^^^^^^^^^^^^^^^^^^
2024-09-05T18:45:00.808+05:30 [APP/PROC/WEB/1] [ERR] ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:2427)
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] ERROR:pika.adapters.utils.io_services_utils:Wrapped func exited with exception. Caller's stack:
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] Traceback (most recent call last):
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/diagnostic_utils.py", line 53, in log_exception_func_wrap
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] return func(*args, **kwargs)
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] ^^^^^^^^^^^^^^^^^^^^^
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/utils/io_services_utils.py", line 1308, in _produce
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] super(_AsyncSSLTransport, self)._produce()
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/utils/io_services_utils.py", line 819, in _produce
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] num_bytes_sent = self._sigint_safe_send(self._sock,
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/utils/io_services_utils.py", line 79, in retry_sigint_wrap
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] return func(*args, **kwargs)
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] ^^^^^^^^^^^^^^^^^^^^^
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/utils/io_services_utils.py", line 861, in _sigint_safe_send
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] return sock.send(data)
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] ^^^^^^^^^^^^^^^
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/ssl.py", line 1242, in send
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] return self._sslobj.write(data)
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] ^^^^^^^^^^^^^^^^^^^^^^^^
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:2427)
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] Traceback (most recent call last):
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/diagnostic_utils.py", line 53, in log_exception_func_wrap
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] return func(*args, **kwargs)
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] ^^^^^^^^^^^^^^^^^^^^^
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/utils/io_services_utils.py", line 1308, in _produce
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] super(_AsyncSSLTransport, self)._produce()
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/utils/io_services_utils.py", line 819, in _produce
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] num_bytes_sent = self._sigint_safe_send(self._sock,
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/utils/io_services_utils.py", line 79, in retry_sigint_wrap
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] return func(*args, **kwargs)
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] ^^^^^^^^^^^^^^^^^^^^^
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/utils/io_services_utils.py", line 861, in _sigint_safe_send
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] return sock.send(data)
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] ^^^^^^^^^^^^^^^
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/ssl.py", line 1242, in send
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] return self._sslobj.write(data)
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] ^^^^^^^^^^^^^^^^^^^^^^^^
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:2427)
2024-09-05T18:45:00.810+05:30 [APP/PROC/WEB/1] [ERR] ERROR:pika.adapters.base_connection:connection_lost: StreamLostError: ("Stream connection lost: SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2427)')",)
2024-09-05T18:45:00.811+05:30 [APP/PROC/WEB/1] [ERR] ERROR:pika.adapters.blocking_connection:Unexpected connection close detected: StreamLostError: ("Stream connection lost: SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2427)')",)
2024-09-05T18:45:00.815+05:30 [APP/PROC/WEB/1] [ERR] DEBUG:db:RabbitMQ Stopped:,Stream connection lost: SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2427)')
2024-09-05T18:45:00.816+05:30 [APP/PROC/WEB/1] [ERR] Traceback (most recent call last):
2024-09-05T18:45:00.817+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/app/data_loader/receiver.py", line 55, in consumer
2024-09-05T18:45:00.817+05:30 [APP/PROC/WEB/1] [ERR] channel.start_consuming()
2024-09-05T18:45:00.817+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 1883, in start_consuming
2024-09-05T18:45:00.817+05:30 [APP/PROC/WEB/1] [ERR] self._process_data_events(time_limit=None)
2024-09-05T18:45:00.817+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 2044, in _process_data_events
2024-09-05T18:45:00.817+05:30 [APP/PROC/WEB/1] [ERR] self.connection.process_data_events(time_limit=time_limit)
2024-09-05T18:45:00.817+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 842, in process_data_events
2024-09-05T18:45:00.817+05:30 [APP/PROC/WEB/1] [ERR] self._flush_output(common_terminator)
2024-09-05T18:45:00.817+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 523, in _flush_output
2024-09-05T18:45:00.817+05:30 [APP/PROC/WEB/1] [ERR] raise self._closed_result.value.error
2024-09-05T18:45:00.818+05:30 [APP/PROC/WEB/1] [ERR] pika.exceptions.StreamLostError: Stream connection lost: SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2427)')
2024-09-05T18:45:00.818+05:30 [APP/PROC/WEB/1] [OUT] None
2024-09-05T18:45:00.818+05:30 [APP/PROC/WEB/1] [ERR] Exception in thread Consumer:
2024-09-05T18:45:00.818+05:30 [APP/PROC/WEB/1] [ERR] Traceback (most recent call last):
2024-09-05T18:45:00.818+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/app/data_loader/receiver.py", line 55, in consumer
2024-09-05T18:45:00.818+05:30 [APP/PROC/WEB/1] [ERR] channel.start_consuming()
2024-09-05T18:45:00.818+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 1883, in start_consuming
2024-09-05T18:45:00.819+05:30 [APP/PROC/WEB/1] [ERR] self._process_data_events(time_limit=None)
2024-09-05T18:45:00.819+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 2044, in _process_data_events
2024-09-05T18:45:00.821+05:30 [APP/PROC/WEB/1] [ERR] self.connection.process_data_events(time_limit=time_limit)
2024-09-05T18:45:00.821+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 842, in process_data_events
2024-09-05T18:45:00.821+05:30 [APP/PROC/WEB/1] [ERR] self._flush_output(common_terminator)
2024-09-05T18:45:00.822+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 523, in _flush_output
2024-09-05T18:45:00.822+05:30 [APP/PROC/WEB/1] [ERR] raise self._closed_result.value.error
2024-09-05T18:45:00.822+05:30 [APP/PROC/WEB/1] [ERR] pika.exceptions.StreamLostError: Stream connection lost: SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2427)')
2024-09-05T18:45:00.822+05:30 [APP/PROC/WEB/1] [ERR] During handling of the above exception, another exception occurred:
2024-09-05T18:45:00.822+05:30 [APP/PROC/WEB/1] [ERR] Traceback (most recent call last):
2024-09-05T18:45:00.822+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/threading.py", line 1045, in _bootstrap_inner
2024-09-05T18:45:00.823+05:30 [APP/PROC/WEB/1] [ERR] self.run()
2024-09-05T18:45:00.823+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/threading.py", line 982, in run
2024-09-05T18:45:00.824+05:30 [APP/PROC/WEB/1] [ERR] self._target(*self._args, **self._kwargs)
2024-09-05T18:45:00.824+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/app/data_loader/receiver.py", line 59, in consumer
2024-09-05T18:45:00.824+05:30 [APP/PROC/WEB/1] [ERR] channel.stop_consuming()
2024-09-05T18:45:00.824+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 1896, in stop_consuming
2024-09-05T18:45:00.825+05:30 [APP/PROC/WEB/1] [ERR] self._cancel_all_consumers()
2024-09-05T18:45:00.826+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 1497, in _cancel_all_consumers
2024-09-05T18:45:00.833+05:30 [APP/PROC/WEB/1] [ERR] self.basic_cancel(consumer_tag)
2024-09-05T18:45:00.833+05:30 [APP/PROC/WEB/1] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 1774, in basic_cancel
2024-09-05T18:45:00.833+05:30 [APP/PROC/WEB/1] [ERR] assert (consumer_info.cancelled_by_broker or
2024-09-05T18:45:00.833+05:30 [APP/PROC/WEB/1] [ERR] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-09-05T18:45:00.833+05:30 [APP/PROC/WEB/1] [ERR] AssertionError: ctag1.fd16c987fd7b4c57a248ee855b306cd8
Log - 3
[ERR] ERROR:pika.adapters.utils.io_services_utils:Socket EOF; <ssl.SSLSocket fd=16, family=2, type=1, proto=6, laddr=('172.28.101.105', 52232)>
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] ERROR:pika.adapters.utils.io_services_utils:Wrapped func exited with exception. Caller's stack:
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] Traceback (most recent call last):
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/diagnostic_utils.py", line 53, in log_exception_func_wrap
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] return func(*args, **kwargs)
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] ^^^^^^^^^^^^^^^^^^^^^
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/utils/io_services_utils.py", line 1235, in _consume
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] super(_AsyncSSLTransport, self)._consume()
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/utils/io_services_utils.py", line 797, in _consume
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] raise self.RxEndOfFile()
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] pika.adapters.utils.io_services_utils._AsyncTransportBase.RxEndOfFile: [Errno -1] End of input stream (EOF)
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] Traceback (most recent call last):
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/diagnostic_utils.py", line 53, in log_exception_func_wrap
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] return func(*args, **kwargs)
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] ^^^^^^^^^^^^^^^^^^^^^
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/utils/io_services_utils.py", line 1235, in _consume
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] super(_AsyncSSLTransport, self)._consume()
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/utils/io_services_utils.py", line 797, in _consume
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] raise self.RxEndOfFile()
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] pika.adapters.utils.io_services_utils._AsyncTransportBase.RxEndOfFile: [Errno -1] End of input stream (EOF)
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] ERROR:pika.adapters.base_connection:connection_lost: StreamLostError: ("Stream connection lost: RxEndOfFile(-1, 'End of input stream (EOF)')",)
2024-09-06T15:58:30.724+05:30 [APP/PROC/WEB/0] [ERR] ERROR:pika.adapters.blocking_connection:Unexpected connection close detected: StreamLostError: ("Stream connection lost: RxEndOfFile(-1, 'End of input stream (EOF)')",)
2024-09-06T15:58:30.724+05:30 [APP/PROC/WEB/0] [OUT] inner rabbit exception
2024-09-06T15:58:30.725+05:30 [APP/PROC/WEB/0] [ERR] Traceback (most recent call last):
2024-09-06T15:58:30.726+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/app/data_loader/receiver.py", line 104, in consumer
2024-09-06T15:58:30.726+05:30 [APP/PROC/WEB/0] [ERR] channel.start_consuming()
2024-09-06T15:58:30.726+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 1883, in start_consuming
2024-09-06T15:58:30.726+05:30 [APP/PROC/WEB/0] [ERR] self._process_data_events(time_limit=None)
2024-09-06T15:58:30.726+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 2044, in _process_data_events
2024-09-06T15:58:30.726+05:30 [APP/PROC/WEB/0] [ERR] self.connection.process_data_events(time_limit=time_limit)
2024-09-06T15:58:30.726+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 842, in process_data_events
2024-09-06T15:58:30.726+05:30 [APP/PROC/WEB/0] [ERR] self._flush_output(common_terminator)
2024-09-06T15:58:30.726+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 523, in _flush_output
2024-09-06T15:58:30.726+05:30 [APP/PROC/WEB/0] [ERR] raise self._closed_result.value.error
2024-09-06T15:58:30.726+05:30 [APP/PROC/WEB/0] [ERR] pika.exceptions.StreamLostError: Stream connection lost: RxEndOfFile(-1, 'End of input stream (EOF)')
2024-09-06T15:58:30.726+05:30 [APP/PROC/WEB/0] [OUT] None
2024-09-06T15:58:30.726+05:30 [APP/PROC/WEB/0] [OUT] main rabbit exception
2024-09-06T15:58:30.726+05:30 [APP/PROC/WEB/0] [ERR] ERROR:pika.adapters.blocking_connection:BlockingConnection.close(200, 'Normal shutdown') called on closed connection.
2024-09-06T15:58:30.727+05:30 [APP/PROC/WEB/0] [ERR] Traceback (most recent call last):
2024-09-06T15:58:30.727+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/app/data_loader/receiver.py", line 116, in consumer
2024-09-06T15:58:30.727+05:30 [APP/PROC/WEB/0] [ERR] connection.close()
2024-09-06T15:58:30.727+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 800, in close
2024-09-06T15:58:30.727+05:30 [APP/PROC/WEB/0] [ERR] raise exceptions.ConnectionWrongStateError(msg)
2024-09-06T15:58:30.727+05:30 [APP/PROC/WEB/0] [ERR] pika.exceptions.ConnectionWrongStateError: BlockingConnection.close(200, 'Normal shutdown') called on closed connection.
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] ERROR:pika.adapters.utils.io_services_utils:Wrapped func exited with exception. Caller's stack:
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] Traceback (most recent call last):
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/diagnostic_utils.py", line 53, in log_exception_func_wrap
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] return func(*args, **kwargs)
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] ^^^^^^^^^^^^^^^^^^^^^
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/utils/io_services_utils.py", line 1235, in _consume
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] super(_AsyncSSLTransport, self)._consume()
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/utils/io_services_utils.py", line 797, in _consume
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] raise self.RxEndOfFile()
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] pika.adapters.utils.io_services_utils._AsyncTransportBase.RxEndOfFile: [Errno -1] End of input stream (EOF)
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] Traceback (most recent call last):
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/diagnostic_utils.py", line 53, in log_exception_func_wrap
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] return func(*args, **kwargs)
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] ^^^^^^^^^^^^^^^^^^^^^
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/utils/io_services_utils.py", line 1235, in _consume
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] super(_AsyncSSLTransport, self)._consume()
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/utils/io_services_utils.py", line 797, in _consume
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] raise self.RxEndOfFile()
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] pika.adapters.utils.io_services_utils._AsyncTransportBase.RxEndOfFile: [Errno -1] End of input stream (EOF)
2024-09-06T15:58:30.723+05:30 [APP/PROC/WEB/0] [ERR] ERROR:pika.adapters.base_connection:connection_lost: StreamLostError: ("Stream connection lost: RxEndOfFile(-1, 'End of input stream (EOF)')",)
2024-09-06T15:58:30.724+05:30 [APP/PROC/WEB/0] [ERR] ERROR:pika.adapters.blocking_connection:Unexpected connection close detected: StreamLostError: ("Stream connection lost: RxEndOfFile(-1, 'End of input stream (EOF)')",)
2024-09-06T15:58:30.724+05:30 [APP/PROC/WEB/0] [OUT] inner rabbit exception
2024-09-06T15:58:30.725+05:30 [APP/PROC/WEB/0] [ERR] Traceback (most recent call last):
2024-09-06T15:58:30.726+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/app/data_loader/receiver.py", line 104, in consumer
2024-09-06T15:58:30.726+05:30 [APP/PROC/WEB/0] [ERR] channel.start_consuming()
2024-09-06T15:58:30.726+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 1883, in start_consuming
2024-09-06T15:58:30.726+05:30 [APP/PROC/WEB/0] [ERR] self._process_data_events(time_limit=None)
2024-09-06T15:58:30.726+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 2044, in _process_data_events
2024-09-06T15:58:30.726+05:30 [APP/PROC/WEB/0] [ERR] self.connection.process_data_events(time_limit=time_limit)
2024-09-06T15:58:30.726+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 842, in process_data_events
2024-09-06T15:58:30.726+05:30 [APP/PROC/WEB/0] [ERR] self._flush_output(common_terminator)
2024-09-06T15:58:30.726+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 523, in _flush_output
2024-09-06T15:58:30.726+05:30 [APP/PROC/WEB/0] [ERR] raise self._closed_result.value.error
2024-09-06T15:58:30.726+05:30 [APP/PROC/WEB/0] [ERR] pika.exceptions.StreamLostError: Stream connection lost: RxEndOfFile(-1, 'End of input stream (EOF)')
2024-09-06T15:58:30.726+05:30 [APP/PROC/WEB/0] [OUT] None
2024-09-06T15:58:30.726+05:30 [APP/PROC/WEB/0] [OUT] main rabbit exception
2024-09-06T15:58:30.726+05:30 [APP/PROC/WEB/0] [ERR] ERROR:pika.adapters.blocking_connection:BlockingConnection.close(200, 'Normal shutdown') called on closed connection.
2024-09-06T15:58:30.727+05:30 [APP/PROC/WEB/0] [ERR] Traceback (most recent call last):
2024-09-06T15:58:30.727+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/app/data_loader/receiver.py", line 116, in consumer
2024-09-06T15:58:30.727+05:30 [APP/PROC/WEB/0] [ERR] connection.close()
2024-09-06T15:58:30.727+05:30 [APP/PROC/WEB/0] [ERR] File "/home/vcap/deps/0/python/lib/python3.11/site-packages/pika/adapters/blocking_connection.py", line 800, in close
2024-09-06T15:58:30.727+05:30 [APP/PROC/WEB/0] [ERR] raise exceptions.ConnectionWrongStateError(msg)
2024-09-06T15:58:30.727+05:30 [APP/PROC/WEB/0] [ERR] pika.exceptions.ConnectionWrongStateError: BlockingConnection.close(200, 'Normal shutdown') called on closed connection.
I attached 3 logs, and these logs are coming randomly.
Luke Bakken
Sep 11, 2024, 10:06:56 AM9/11/24
to Pika
Hello,
I requested that you use GitHub discussions to continue -
https://github.com/pika/pika/discussions
Start a new discussion there. Please take the time to ensure your Pika logs are formatted in such a way that they are easy to read. I don't have time to reformat your logs for you.
Start a new discussion there. Please take the time to ensure your Pika logs are formatted in such a way that they are easy to read. I don't have time to reformat your logs for you.
In addition, I requested access from my personal email to that google drive.
Again, start a discussion here - https://github.com/pika/pika/discussions
Thanks,
Luke
On Wednesday, September 11, 2024 at 5:15:06 AM UTC-7 advance...@mercedes-benz.com wrote:
Hi Luke,Please find the required details below:Python version - 3.11.9
Pika version - 1.3.2RabbitMQ logs - RabbitMQ Logs
Reply all
Reply to author
Forward
0 new messages