How to trace messages from Kafka or others technologies.

[Guillermo Ortiz]

Hello, 

I was reading documentation about Zipkin and although I have an "light" idea how it works, I have a some doubts.

I have been taking a look to Brave and Finagle, they have Instrumentations to MySQL, JAX, HTTP and so on. if I undertood well, you have to change your code and introduce some code to use HttpFilters or similar, so it's not possible to make it really without code?

If I was right, how could I trace messages from Kafka for example? Are there any instrument built to do it?

I'm not sure how I could trace those messages.

Should I modified the code in the kafkaClient? how about the Kafka side? 

I'm talking about Kafka, but I have the same doubts for example with a Queue, Flume or messages that go inside HBase and so on. 

[Kristof Adriaenssens]

Hello Guillermo,

All the Zipkin compatible distributed tracing implementations like Brave and Finagle need 'code changes' at client side (outgoing requests) and server side (incoming requests).

Because most frameworks are extensible and have the possibility to configure interceptors at client or server side (jaxrs2, jersey2, okhttp,...) the code change are typically just configuring an already existing interceptor.

There are some solutions like Naver/pinpoint [0] which use an agent based approach for instrumentation. This typically does not require code changes but relies on an additional agent process.

When you saw Kafka in the context of OpenZipkin it is probably about using Kafka to transport trace data for eventual storage in a datastore. Zipkin is today typically used to instrument RPC style communication and not so much for tracing messaging systems. However that might change in future and in fact Spring Cloud Sleuth / Spring Cloud Zipkin has support for tracing messaging systems [1]. 

[0] : https://github.com/naver/pinpoint

[1] : https://spring.io/blog/2016/02/15/distributed-tracing-with-spring-cloud-sleuth-and-spring-cloud-zipkin

Op maandag 6 juni 2016 14:24:12 UTC+2 schreef Guillermo Ortiz:

[宋鑫]

There are two methods.First method,before sending msg to Kafka,you should generate a span and use it as a msg key or wrap your original msg with this span, then send to kafka.

In kafka consumer,you get the span and manual trigger a ServerRequestReceive action,code as below:

//you just convert your span to a HTTPREQUEST
brave.serverRequestInterceptor().handle(new HttpServerRequestAdapter(new MyHttpServerRequest(myRequest), new MySpanNameProvider("receive-from-kafka")));

//Or maybe this,it's eeasyeasyieasyeasieeasier

brave.serverTracer().setStateCurrentTrace(span,"spanName");
//Then you manual trigger  a ServerResponseReceive,then brave REPORT the span.
brave.serverRequestInterceptor().handle(new HttpServerRequestAdapter(new MyHttpServerRequest(myRequest), new MySpanNameProvider("receive-from-kafka")))

Second method,if you don't want wrap the Span,you can binde Span to kafka msg key,and store it to Redis.Then ,when you consume from kafka,you use msg key and query if there is Span in redis.If not ,not trace;if yes,get the Span,and binde this span to currentSpan,

then manual trigger a HttpServerResponse，or use a easier way.

brave.serverTracer().setServerReceived();

[Adrian Cole]

thanks. very nice! ps the more recent version of brave (4) has a nicer
api for async tracing. here's an example
https://gist.github.com/adriancole/76d94054b77e3be338bd75424ca8ba30

> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "zipkin-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to zipkin-user...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.