Inconsistent ParseFromArray return value from the same message sometime return false sometime return true

[hce h]

Hi,

I am running protobuf C++ on Debian Jessie, most of the messages work fine, but one message is not stable, ParseFromArray from the same message sometime return false sometime return true, the protobuf library complains string tokenRef invalid non-UTF-8 characters. It worries me the inconsistent return value from the ParseFromArray, if the size was wrong or if the invalid non-UTF-8 characters caused the problems, why sometime it can be passed to return true? Has anyone seen this behavior?

Thanks for your comments in advance.

[Adam Cozzette]

If ParseFromArray() is returning different results for the same serialized proto then that does sound like a bug. Do you have an example of a proto that triggers this behavior? Also, what version of the protobuf compiler and library are you using? I suspect Debian might be packaging an older version of it and so if you're using the package that comes with Debian then that might be a bit stale.

It might also be worth double-checking that you're passing the right size to ParseFromArray(); I can imagine that if you pass a size that's a few bytes too large then you would get unpredictable results as the code tries to parse beyond the end of the actual proto data.

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscribe@googlegroups.com.
To post to this group, send email to prot...@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

[hce h]

The version is libprotobuf9:armhf  2.6.1-1, it does not look like an older version. I am certain the problem is not the size, I am aware of that the wrong size could cause the problem. The protobuf library did not like some characters, if I put some string filed with empty string, it would work in most of times, but that is also not consistent, some time it could fail again.

Is there anyway to display google libprotobuf error message when ParseFromArray failed? It is very hard to debug the message before failing on deserialization. If you could share me tricks how to debug protobuf message when failing on ParseFromArray, I'll much appreciate it.

Thank you.

On Wednesday, February 15, 2017 at 2:06:53 AM UTC+11, Adam Cozzette wrote:

If ParseFromArray() is returning different results for the same serialized proto then that does sound like a bug. Do you have an example of a proto that triggers this behavior? Also, what version of the protobuf compiler and library are you using? I suspect Debian might be packaging an older version of it and so if you're using the package that comes with Debian then that might be a bit stale.

It might also be worth double-checking that you're passing the right size to ParseFromArray(); I can imagine that if you pass a size that's a few bytes too large then you would get unpredictable results as the code tries to parse beyond the end of the actual proto data.

On Fri, Feb 10, 2017 at 2:44 AM, hce h <jupit...@gmail.com> wrote:

Hi,

I am running protobuf C++ on Debian Jessie, most of the messages work fine, but one message is not stable, ParseFromArray from the same message sometime return false sometime return true, the protobuf library complains string tokenRef invalid non-UTF-8 characters. It worries me the inconsistent return value from the ParseFromArray, if the size was wrong or if the invalid non-UTF-8 characters caused the problems, why sometime it can be passed to return true? Has anyone seen this behavior?

Thanks for your comments in advance.

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+u...@googlegroups.com.

[Adam Cozzette]

If you build in debug mode then that should enable some assertions that could potentially help. Doing that and then looking at the logs is probably the best way to get more debug information.

If you could share a short snippet of code then that would help to track down the problem, too.

To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscribe@googlegroups.com.

[hce h]

Thanks Adam, please see embedded comments.

On Thursday, February 16, 2017 at 5:08:59 AM UTC+11, Adam Cozzette wrote:

If you build in debug mode then that should enable some assertions that could potentially help. Doing that and then looking at the logs is probably the best way to get more debug information.

My program was built in debug, but I'd like to see the google/protobuf library error message like following something I only saw once when it failed on ParseFromArray, which gave me a clue and I replaced the string tokenref by empty string, then it worked.

[libprotobuf ERROR google/protobuf/wire_format.cc:1091] String field 'tokenref' contains invalid UTF-8 data when parsing a protocol buffer. Use the 'bytes' type if you intend to send raw bytes.

The library was installed from apt-get and I don't see any google/protobuf library logs in /var/log. Or did I misunderstand your comments?

If you could share a short snippet of code then that would help to track down the problem, too.

message Request {
    required string userName = 1;
    required string tokenRef = 2;
}
 
Request request;
if (request.ParseFromArray(buffer, length)) {
    doProcess(request);
}

The string tokenRef contains real address and other information which I could not show you, but that string is pure English characters plus "-" and digital numbers. If I put that string to an empty string, it would work. My concern is not the string I need to fix, but rather my capacity to debug the failing on ParseFromArray, without showing me once from above google library error message "libprotobuf ERROR google/protobuf/wire_format.cc...", I would have no clues to fix it.

So my question again, how can I see libprotobuf error or log messages?

Thank you.

[Adam Cozzette]

The kind of log you posted is what I was talking about; these logs don't appear anywhere in /var/log but just get printed to stderr.

To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscribe@googlegroups.com.