Using grpc_completion_queue to enqueue non-grpc events

[sut...@gmail.com]

Hi folks,

I'm wondering if there's a way in grpc::CompletionQueue to insert events dequeued from a non-grpc job queue? 

I've two queues in my server: one grpc::CompleteQueue and another unrelated internal queue. Currently, I'm polling both queues in the same thread for availability of an event. It sort of busy loops. I would like to have one stop for all event multiplexing. If there is a way to insert an event into a grpc_completion_queue (with some tag) and retrieve it back out from cq_.AsynNext, that would do the trick. Effectively, use gprc_completion_queue as a pipe. Is this possible?

A very similar question was discussed here: https://groups.google.com/forum/#!topic/grpc-io/NrrPOKEZbAU Looks like back in 2016 no such way existed.

Alternatively, is there a way to notify a condition_variable upon availability of an event in the grpc_completion_queue? The polling thread would block simply block on the condition_variable and will subsequently call cq.AsyncNext with zero timeout.

Regards,

Sumant

[sre...@gmail.com]

Hi Sumant,

Unfortunately, the answer to the previous email thread you mentioned (i.e  https://groups.google.com/forum/#!topic/grpc-io/NrrPOKEZbAU ) is still relevant.

There is currently no way to do that although in theory it should be possible to (by exposing the low-level/internal completion queue APIs i.e grpc_cq_begin_op() and grpc_cq_end_op() APIs via C++ wrappers).  There isn't a way to signal a condition variable either 

If possible, do create a ticket so that the team can triage it.

-Sree

[Vijay Pai]

Hi Sree and Sumant,

So, it's a little subtle but there is API to inject events into the C++ completion queue: just set a grpc::Alarm with an immediate deadline. That will post the tag that you specify right away.

- Vijay

--
You received this message because you are subscribed to the Google Groups "grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+u...@googlegroups.com.
To post to this group, send email to grp...@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/d38eac53-c71e-4d00-b3ee-300719ead390%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[sre...@gmail.com]

That totally makes sense. Thanks for the suggestion Vijay!

[Pau Freixes]

Hi,

Regarding the last queation on using a conditional variable, If Im not wrong the new experimental callback complwtio  queue should give you enought freedom for implementing what you are looking for - Vijay as an author will correct me

On Thu, Feb 7, 2019, 07:06 <sre...@gmail.com wrote:

That totally makes sense. Thanks for the suggestion Vijay!

--
You received this message because you are subscribed to the Google Groups "grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+u...@googlegroups.com.
To post to this group, send email to grp...@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.

To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/5a61674a-b7b8-49b5-9314-ada6b4382502%40googlegroups.com.

[sut...@gmail.com]

Hi Sree, Vijay, and Pau,

All the suggestions above look promising. 

grpc::Alarm is closest to what I'm looking for. I'll give it a shot. What may be the consequences of posting immediate deadline alarms at a high rate? Possibly 10k per second. In my setup that's the throughput of the "other" queue. Cost of alarms may be reduced by "batching" multiple queue items into an alarm. 

Sree, you mentioned grpc_cq_begin_op() and grpc_cq_end_op() APIs. I see that C++ gprc::CompletionQueue::cq() returns a pointer to the underlying grpc_completion_queue. That should allow us to play games with begin/end ops using the C and C++ api simultaneously, No? Are there gotchas?

Pau, thanks for bringing up the callback api. What is the timeline for it to be official or even beta?

Regards,

Sumant 

[Vijay Pai]

I just created a pull request to benchmark immediately-firing alarms. The results on my Mac laptop suggest that you can fire at least 3,000,000 immediate alarms per second on a single thread, so I don't think you'll hit a limit from that.

I would strongly advise against directly using grpc_cq_begin_op/grpc_cq_end_op . Those are internal functions and will change name/behavior whenever we choose.

I should have some more information to announce on the callback API within the next few weeks but we continue to prepare it for production-readiness in both unary and streaming cases.

- Vijay

[sut...@gmail.com]

Hi Vijay,

3,000,000 per second rocks!

Thx much everyone!

Regards,

Sumant

[sut...@gmail.com]

Hi Vijay,

Just confirming that FIFO ordering of immediately-firing alarms is guaranteed by the complete_queue. Right?

-Sumant 

[Sree Kuchibhotla]

Hi Sumant,

If you create two alarms with zero timeout in quick succession, then they may not "fire" in the same order you created - and hence FIFO isn't guaranteed here. However, if the alarms are created reasonably spaced apart (like for example: tens of millis apart), then yes -they fire in FIFO.

(Note that an alarm underneath creates a timer which when fired queues the completion event in the completion queue. So we are really at the mercy of the grpc timer component.)..

Hope this helps.

-Sree

--
You received this message because you are subscribed to a topic in the Google Groups "grpc.io" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/grpc-io/NyjgbqHxTm8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to grpc-io+u...@googlegroups.com.

To post to this group, send email to grp...@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.

To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/98abb701-1b90-4c2f-87ea-821b83d17564%40googlegroups.com.

[sut...@gmail.com]

Hi Sree,

Whoa! This behavior is such a gotcha. It seem you are saying immediately-firing alarms don't in fact "fire immediately", Right? Naively one would think that zero-deadline alarms would be optimized and would be queued right-away as there's no apparent reason for using a timer. That's just me though.

Is it worth adding such an optimization in gRPC core?

Thx a bunch for answering such a nuanced question promptly.

Regards,

Sumant

[Sree Kuchibhotla]

Hi Sumant,

Zero timeout alarms is not a common case (Vijay can correct me here) and so it wouldn't make sense to special-case this.

-Sree

To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/921b1360-8f74-484f-af53-6755715d9708%40googlegroups.com.

[Vijay Pai]

Sorry for the late response. To further what Sree said, any use of alarms at all is not a common case so we've never special cased any of this. I imagine that we could accept a PR but I also expect that it would have to pass the "but why" threshold. None of this is what we consider timing-sensitive as long as it's good enough. And to confirm Sree: nothing is guaranteed in-order at the CQ, regardless of timeout.

[Sumant Tambe]

Can you please elaborate on "nothing is guaranteed in-order at the CQ"? Is it wrong to think about the CQ as a queue?

Reiterating the original objective: to use grpc completion_queue to tunnel non-grpc events so that the application event loop has only one queue to wait on. The FIFO order of non-grpc events must be preserved.

Sound like alarms is not the right path. "But Why" question is harder to answer if Alarm is the answer. 

An alternative is grpc_cq_begin_op() and grpc_cq_end_op() APIs. However, it looks like applications using this api directly is not desirable for portability reasons. Is it desirable to the gRPC team to extend C or C++ completion_queue design/implementation to facilitate insertion of application-defined completed ops/tags?

I'm not sure how the two approaches compare with respect to complexity, intrusiveness, and hackiness spectrum. Thoughts?

To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/cb3be957-4d35-4fbb-9f4e-2f94bffd7088%40googlegroups.com.

[Vijay Pai]

Sorry for the delayed response. The CQ shouldn't be thought of as a FIFO queue primarily largely because in a highly-concurrent system like gRPC the definition of "first" is not very clear. These different alarms might have been resolved at and fired from different internal threads, for example, so they might hit the CQ in a different order than they were first put into the system. If you need to preserve FIFO order, you should probably do that by tagging your events with some sort of atomically incrementing value (for example) and then expecting those in sequence when pulling tags off the CQ.