Exporting Jaeger reported latencies to json/xml file

[Aditi Ghag]

Are there any ways to export Jaeger reported latencies for API calls made between microservices to an xml or a json file?

[Won Jun Jang]

This doesn't exist at the moment. Can you let us know what you're attempting to do and we can provide assistance. Are you interested in only the network latency?

On Tue, Sep 4, 2018 at 1:28 PM Aditi Ghag <aditi...@gmail.com> wrote:

Are there any ways to export Jaeger reported latencies for API calls made between microservices to an xml or a json file?

--
You received this message because you are subscribed to the Google Groups "Jaeger Tracing" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jaeger-tracin...@googlegroups.com.
To post to this group, send email to jaeger-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jaeger-tracing/a7b34539-805f-4321-9026-9464a83e7926%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Yuri Shkuro]

> This doesn't exist at the moment. 

well, once the trace is saved in the Jaeger backend, you can get its JSON representation from the query service.

On Tue, Sep 4, 2018 at 2:05 PM, 'Won Jun Jang' via Jaeger Tracing <jaeger-...@googlegroups.com> wrote:

This doesn't exist at the moment. Can you let us know what you're attempting to do and we can provide assistance. Are you interested in only the network latency?

On Tue, Sep 4, 2018 at 1:28 PM Aditi Ghag <aditi...@gmail.com> wrote:

Are there any ways to export Jaeger reported latencies for API calls made between microservices to an xml or a json file?

--
You received this message because you are subscribed to the Google Groups "Jaeger Tracing" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jaeger-tracing+unsubscribe@googlegroups.com.
To post to this group, send email to jaeger-tracing@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jaeger-tracing/a7b34539-805f-4321-9026-9464a83e7926%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "Jaeger Tracing" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jaeger-tracing+unsubscribe@googlegroups.com.
To post to this group, send email to jaeger-tracing@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jaeger-tracing/CAHdEcWyNrUm3%3DMEgycEhQY8X5Kpd-7et_LTCnKt_RP8VEK407A%40mail.gmail.com.

[Aditi Ghag]

Thanks for your reply.

We need to run some load tests on applications, and get latency results for all the queries by the load tests. 

It'll be nice to get the break down of the latency into application and network induced latencies. But I don't think it's possible to decouple the two in the existing framework, is it?

[Yuri Shkuro]

When service A calls service B, service A records a "client" span, and B records a "server" span.

The difference between in duration between these two spans are the network latency.

To unsubscribe from this group and stop receiving emails from it, send an email to jaeger-tracing+unsubscribe@googlegroups.com.
To post to this group, send email to jaeger-tracing@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jaeger-tracing/f130344d-10e7-44c4-8697-0724256b1f9b%40googlegroups.com.

[Aditi Ghag]

Re-posting my previous messages. 

We need to run some load tests on applications, and get latency results for all the queries by the load tests. 

How do we differentiate between latency query results from various runs of the load tests?

On Tuesday, September 4, 2018 at 2:30:22 PM UTC-7, Yuri Shkuro wrote:

> This doesn't exist at the moment. 

well, once the trace is saved in the Jaeger backend, you can get its JSON representation from the query service.

On Tue, Sep 4, 2018 at 2:05 PM, 'Won Jun Jang' via Jaeger Tracing <jaeger-...@googlegroups.com> wrote:

This doesn't exist at the moment. Can you let us know what you're attempting to do and we can provide assistance. Are you interested in only the network latency?

On Tue, Sep 4, 2018 at 1:28 PM Aditi Ghag <aditi...@gmail.com> wrote:

Are there any ways to export Jaeger reported latencies for API calls made between microservices to an xml or a json file?

--
You received this message because you are subscribed to the Google Groups "Jaeger Tracing" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jaeger-tracin...@googlegroups.com.
To post to this group, send email to jaeger-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jaeger-tracing/a7b34539-805f-4321-9026-9464a83e7926%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jaeger Tracing" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jaeger-tracin...@googlegroups.com.
To post to this group, send email to jaeger-...@googlegroups.com.

[Yuri Shkuro]

> How do we differentiate between latency query results from various runs of the load tests?

You can tag the traces with any custom information that you need. Having said that, since these are load tests, why not just use a metrics system to track the latency? Traces are useful if you want to investigate the reasons for latency, not simply observe it as part of the load test.

To unsubscribe from this group and stop receiving emails from it, send an email to jaeger-tracing+unsubscribe@googlegroups.com.
To post to this group, send email to jaeger-tracing@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jaeger-tracing/21de728a-5713-4007-b4a9-8dd541620afe%40googlegroups.com.

[Aditi Ghag]

It's not straight forward to tag the traces, the load test can generate thousands of calls to the microservices. 

We are tracking the overall end-to-end application request latency along with the latency between various microservices the application request traverses through. How do we get this information using a metrics system? Essentially, we want to find out which microservices calls are contributing to the overall application request latency the most, and how they vary with the increasing load. 

[Yuri Shkuro]

If you plan to analyze the latency in the context of a complete call graph, then yes, it's better done with tracing data.

> It's not straight forward to tag the traces, the load test can generate thousands of calls to the microservices. 

why not? what do you use to generate the load? you can either start the trace from the load generator and tag the trace with some metadata or if the generator is external then perhaps you can add an HTTP header like jaeger-debug-id, which will still tag the traces (but also force sampling).

To unsubscribe from this group and stop receiving emails from it, send an email to jaeger-tracing+unsubscribe@googlegroups.com.
To post to this group, send email to jaeger-tracing@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jaeger-tracing/60e6e180-08c8-4325-b392-7e5fc80e2f47%40googlegroups.com.

[Aditi Ghag]

On Tuesday, September 4, 2018 at 4:58:50 PM UTC-7, Yuri Shkuro wrote:

If you plan to analyze the latency in the context of a complete call graph, then yes, it's better done with tracing data.

> It's not straight forward to tag the traces, the load test can generate thousands of calls to the microservices. 

why not? what do you use to generate the load? you can either start the trace from the load generator and tag the trace with some metadata or if the generator is external then perhaps you can add an HTTP header like jaeger-debug-id, which will still tag the traces (but also force sampling).

Yuri,

Need more details. :)

So we have deployed the sock shop app with istio tracing enabled. Each of the microservice container of the application has an istio-proxy sidecar container running. There is an istio-tracing pod which is an all-in-one Jaeger container. 

- Getting latency information in json 

I see various spans for every microservice and their corresponding latencies in the Jaeger UI. But we need this information exported in json format so that we can collect the results after every experiment and analyze microservices with highest latency values.  What's the url you mentioned in your previous comment to get the json data from query service?

- Tagging load test requests

We are using the loadtest pods that are part of the sock-shop app [1]. How do we add a jaeger debug id to the requests being generated so that we can directly query latency metric for various api calls made to all the microservices? 

Appreciate your help!

Thanks!

[1] https://github.com/microservices-demo/microservices-demo/blob/master/deploy/kubernetes/manifests/loadtest-dep.yaml#L25

 

[Yuri Shkuro]

You can retrieve trace JSON from jaeger query, e.g. https://hostname/api/traces/1e3fe09a9e263fd6

> We are using the loadtest pods that are part of the sock-shop app [1]. How do we add a jaeger debug id to the requests being generated so that we can directly query latency metric for various api calls made to all the microservices? 

args: ["-c", "while true; do locust --host http://front-end.sock-shop.svc.cluster.local -f /config/locustfile.py --clients 5 --hatch-rate 5 --num-request 100 --no-web; done"]

I don't know how much control you have over the requests these thing sends. If you can affect the HTTP headers, you can pass the header "jaeger-debug-id: some-value" that will trigger forced sampling. 

To unsubscribe from this group and stop receiving emails from it, send an email to jaeger-tracing+unsubscribe@googlegroups.com.
To post to this group, send email to jaeger-tracing@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jaeger-tracing/eb3458b0-335a-4a79-a988-d6746b9e09f0%40googlegroups.com.

[Aditi Ghag]

Thanks a lot, Yuri. 

For others, the supported query apis are mentioned here - https://github.com/jaegertracing/jaeger/issues/456