remote_write {"error":"proto: illegal wireType 6"}

[Qiang Ma]

Hi, All,

I am trying to write a Java client to push metrics through remote_write (as I want to use PromQL to narrow down number of metrics).

The receiving end is an InfluxDB and it is taking data from prometheus native "remote_write" with no problem.

But when I use the Java client to post the payload (encoded with  org.xerial.snappy.Snappy)  and the client got this error:

    {"error":"proto: illegal wireType 6"}

There are not much I could find how to trouble shoot to understand what's the problem... hoping there is someone in this group can lend me some tips!

Thanks in advance!

Regards,

Qiang

[Julius Volz]

Are you setting the right headers (esp. "Content-Encoding: snappy") so that the remote end knows the request is snappy-compressed? Like these here:

https://github.com/prometheus/prometheus/blob/cd12f0873c3eb2031f7ba9b2e169449aa1012e3f/storage/remote/client.go#L84-L87

Because your error is a protobuf decoding error referring to a wire type which protobuf itself does not define (max wire type is 5, see https://developers.google.com/protocol-buffers/docs/encoding#structure), maybe the receiving end is trying to interpret the request as protobuf without decompressing it first?

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/b3c9aaab-d85b-4b81-a922-2c2032916aac%40googlegroups.com.

[Qiang Ma]

Thank you for the response!

My code does have the headers set  , see below:

byte[] compressed = Snappy.compress(payload.getBytes("UTF-8"));
LOGGER.log(Level.INFO, "Sending ="+ url + " with: \n"+  new String(Snappy.uncompress(compressed)));
HttpClient client = HttpClient.newHttpClient();
HttpRequest request = HttpRequest.newBuilder()
        .uri(URI.create(url))
        .setHeader("Content-Type", "application/x-protobuf")
        .header("Content-Encoding", "snappy")
        .setHeader("X-Prometheus-Remote-Write-Version", "0.1.0")
        .header("User-Agent", "userAgent")
        .POST(HttpRequest.BodyPublishers.ofByteArray(compressed))
        .build();

Also, thanks for pointing out the wire type 6 is not even listed...  maybe I will set up a different remote_write target to try out.  Or I can contact influx DB to see whether they extended wire type 6 for something else :)

Regards,

Qiang

On Sunday, April 19, 2020 at 2:31:07 PM UTC-5, Julius Volz wrote:

Are you setting the right headers (esp. "Content-Encoding: snappy") so that the remote end knows the request is snappy-compressed? Like these here:

https://github.com/prometheus/prometheus/blob/cd12f0873c3eb2031f7ba9b2e169449aa1012e3f/storage/remote/client.go#L84-L87

Because your error is a protobuf decoding error referring to a wire type which protobuf itself does not define (max wire type is 5, see https://developers.google.com/protocol-buffers/docs/encoding#structure), maybe the receiving end is trying to interpret the request as protobuf without decompressing it first?

On Sun, Apr 19, 2020 at 3:52 PM Qiang Ma <muma...@gmail.com> wrote:

Hi, All,

I am trying to write a Java client to push metrics through remote_write (as I want to use PromQL to narrow down number of metrics).

The receiving end is an InfluxDB and it is taking data from prometheus native "remote_write" with no problem.

But when I use the Java client to post the payload (encoded with  org.xerial.snappy.Snappy)  and the client got this error:

    {"error":"proto: illegal wireType 6"}

There are not much I could find how to trouble shoot to understand what's the problem... hoping there is someone in this group can lend me some tips!

Thanks in advance!

Regards,

Qiang

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to promethe...@googlegroups.com.

[Qiang Ma]

Hi,  some update on this topic,

After studying the protobuf , I found out I can't compress a JSON (beginner's mistake)

The following will generate a valid remote write HTTP post:

1. Download the proto files from https://github.com/prometheus/prometheus/tree/master/prompb (also need goto.proto, and google.protobuf packages, so this can be handy: https://github.com/rgordill/prometheus-kafka-remote-writer-sb/tree/master/src/main/proto )

2. Since I am using java, there is some gradle plugin can help compiling proto file into java: https://github.com/google/protobuf-gradle-plugin

3. Construct a "RemoteWrite" object using those generated java classes (Builder) and use snappy to compress the ByteArray from the RemoteWrite object... post it through the HttpRequest .... Bingo, the remote write went over the wire and it shows up in InfluxDB (it listens at  http://x.x.x.x:8086/api/v1/prom/write?db=prometheus ) A small trick is to set metrics name "up" is to create a Label like ("__name__","up") in the timeseries.

Cheers,

Qiang