first message loss?

[Lior Messinger]

Hi, I'd like to point out a possible message loss, despite the guarantee stated in the documentation. We might have found a way to prevent it but i wondered why the design is as such to allow for this loss.

Here's the scenario

1. time 0:00: consumer on channel A subscribes via nsqlookup, start polling for the topic T
2. time 0:00: topic T is not found, 404
3. time 0:01s: producer message sent to topic, topic created
4. time 0:02s: consumer on channel B looks for topic, receives message
5. time 0:02s: message deleted (nsq thinks there no more consumers)
6. time 0:15s: channel A looks for topic, subscribes (sends RDY 1)

which means, for first document, first message for consumer A will not arrive!

We tried to send a warmup message after each subscription, but that's a problem. not always we have a way to have a producer.
Another option we tried is to use /topic/create post request, to create the topic and channel ahead of time - but it doesnt seem to work

any other ideas?

many thanks
Lior

[Pierce Lopez]

Pre-creating topics and channels in nsqd should work. You can also pre-create topics and channels in nsqlookupd, and nsqadmin has a page/form to do this to all known nsqlookupds. When nsqd receives the first published message for a topic which doesn't exist yet in that nsqd, it queries nsqlookupd to find all existing channels for that topic, so it can pre-create all the appropriate channels and avoid this exact initial-message-loss problem.

- Pierce

--
You received this message because you are subscribed to the Google Groups "nsq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nsq-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nsq-users/c71ce759-05bc-4b61-8634-07d409a3cb9en%40googlegroups.com.