lost messages in Kombu

8 views
Skip to first unread message

Rich Rauenzahn

unread,
Oct 22, 2021, 8:46:53 PM10/22/21
to carrot-users
I was having an issue with periodic lost messages -- I figured out how to "fix" it, but I don't understand *why* it fixes it.  So it leaves me uneasy :)

It turns out in certain cases, I wasn't calling q.close() nor connection.release().   Does it make sense that a q.put() might never get actually sent if these aren't explicitly called?   So that I might understand Kombu+Rabbit better -- why?

(The q and connections objects below otherwise would have been left to the python GC... they are not persistent, they're actually part of a context manager, and the containing function returns promptly.)

Using RabbitMQ, the q setup code was:

        self.no_ack = no_ack
        self.ttl = ttl  # This is set for 7 days.
        self.connection = kombu.Connection(AMQP_URL)
        self.q = SimpleQueue(
            channel=self.connection,
            name=self.qname,
            no_ack=no_ack,
            queue_args={"x-expires": int(self.ttl * 1000)},
            serializer="msgpack",
        )
        self.q.consumer.qos(prefetch_count=1)  # I've set this to 0 now.
        self.q.consumer.accept = prepare_accept_content(["msgpack", "json"])
Reply all
Reply to author
Forward
0 new messages