How to use Zipkin tracing with queuing service?

[Val]

We have a front end service that starts a trace every time a user makes a request. Also, we have a queuing service that accumulates "jobs" - those user requests. On the other side of the equation we have "workers" - processes that periodically check the queue and take "jobs" from it. How to make is so the trace ID that is kept in a "job" as a piece of data is a acquired as a zipkin trace ID for a span initiated by a worker. In other words, how to substitute a trace ID to make the "job" a part of the trace started by a user?

[Adrian Cole]

We have a front end service that starts a trace every time a user makes a request. Also, we have a queuing service that accumulates "jobs" - those user requests. On the other side of the equation we have "workers" - processes that periodically check the queue and take "jobs" from it. How to make is so the trace ID that is kept in a "job" as a piece of data is a acquired as a zipkin trace ID for a span initiated by a worker. In other words, how to substitute a trace ID to make the "job" a part of the trace started by a user?

Sounds like you have a long-running trace in this case. Using instrumentation, you can assert a specific trace id you've stored somewhere.

In other words, this pattern is possible, just it would be instrumentation-specific ( ruby, python, java, scala, .net, etc.. there are tracers that have different features)

I know that we are working towards a standard approach for trace instrumentation, so that use cases like this aren't different per-library, while still being idiomatic to the language of it. http://opentracing.github.io/ 

[Anil Wadghule]

Did you find a solution to this problem? If yes, could you elaborate how did you do this? 

Thank you,

Anil

[Anil Wadghule]

.

On Friday, 7 August 2015 07:25:21 UTC+5:30, Val wrote:

[Adrian Cole]

There are couple ways to do this. If in SPSC model (there's only one
producer of a job and one consumer), you can stash the propagated
context (ex B3 headers) in the message itself. Afterwards, on the
other side, you read these headers the same way an RPC works, except
you have no response. For example, these result in the annotations
"cs", "sr" (client send, client receive), but not "ss", "cr" (server
send, client receive) as there's no backtrace. We call this approach
one-way span, and it is used for example in the brave (java) pending
work for Kafka

https://github.com/openzipkin/brave/pull/461


We have a SPMC (broadcast messaging) approach that works with
potentially multiple recipients via child spans. This also clarifies
some annotations. This work will go into the next version of zipkin,
particularly in support of a simplified json format for new tracers.

https://github.com/openzipkin/zipkin/issues/1654#issuecomment-318546734

Please ask any other questions you like!

> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "zipkin-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to zipkin-user...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.