NullPointerException when using parseFrom

[Jamu]

Hi there, I hope that you are doing well!

I'm encountering some NullPointerException when trying to deserialize my protobuf message using the parseFrom method from my generated class (you can have a look at the stack below). Code has been generated using protoc 3.7.0, and we are currently using protobuf 3.8.0 libraries at runtime.

at com.ds.bus.message.BusMessageProtos$Item.<init>(BusMessageProtos.java:4429)
at com.ds.bus.message.BusMessageProtos$Item$1.parsePartialFrom(BusMessageProtos.java:5317)
at com.ds.bus.message.BusMessageProtos$Item$1.parsePartialFrom(BusMessageProtos.java:5311)
at com.google.protobuf.CodedInputStream$ArrayDecoder.readMessage(CodedInputStream.java:886)
at com.ds.bus.message.BusMessageProtos$Order.<init>(BusMessageProtos.java:3221)
at com.ds.bus.message.BusMessageProtos$Order$1.parsePartialFrom(BusMessageProtos.java:4311)
at com.ds.bus.message.BusMessageProtos$Order$1.parsePartialFrom(BusMessageProtos.java:4305)
at com.google.protobuf.CodedInputStream$ArrayDecoder.readMessage(CodedInputStream.java:886)
at com.ds.bus.message.BusMessageProtos$Message.<init>(BusMessageProtos.java:982)
at com.ds.bus.message.BusMessageProtos$Message$1.parsePartialFrom(BusMessageProtos.java:2127)
at com.ds.bus.message.BusMessageProtos$Message$1.parsePartialFrom(BusMessageProtos.java:2121)
at com.google.protobuf.AbstractParser.parsePartialFrom(AbstractParser.java:100)
at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:120)
at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:125)
at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:48)
at com.ds.bus.message.BusMessageProtos$Message.parseFrom(BusMessageProtos.java:1334)
at com.ds.bus.message.data.ChunkStore.getAndRemoveAllCompleted(ChunkStore.java:93)

NullPointerException are not happening all the time. Usually, it works at the beginning and at some point everything fall apart and all deserialization are ending in a NullPointerException. This issue is only happening in production after some time, I tried to replicate the problem locally and failed to do so.

Could it be an issue fixed in a more recent releases or already seen by someone ?

Any input would be appreciated :)

Thanks in advance for your help!

================== Full proto file

// [START declaration]
syntax = "proto3";

import "google/protobuf/timestamp.proto";
package com.ds.bus.message;
// [END declaration]

// [START options options_declaration]
option java_package = "com.ds.bus.message";
option java_outer_classname = "BusMessageProtos";
option optimize_for = SPEED;
// [END options_declaration]

// [START messages]

message Chunk {
    string identifier = 1;
    uint32 chunk_id = 2;
    uint32 nb_chunks = 3;
    bytes  serialized_message = 4;
}

message Message {
    string identifier = 1;
    oneof order {
        Order simple_order = 2;
        Transaction transaction_order = 3;
        Notification notification = 4;
    }
}

message Transaction {
    string identifier = 1;
    repeated Order orders = 4;
}

message Order {
    string identifier = 1;
    Order.Operation operation = 2;
    enum Operation {
        PUT = 0;
        DELETE = 1;
    }
    map<string, Value> parameters = 3;
    Item item = 4;
}

message Item {
    string identifier = 1;
    string type = 2;
    map<string, Value> attributes = 3;
}

// Supported values for attributes
message Value {
     oneof value {
         double double   = 1;
         float  float    = 2;
         uint32 integer  = 3;
         uint64 long     = 4;
         bool   boolean  = 5;
         string string   = 6;
         bytes  bytes    = 7;
         ValueList list  = 8;
         ValueMap map    = 9;
     }
}

message ValueList {
    repeated Value values = 1;
}

message ValueMap {
    map<string, Value> values = 1;
}

message Notification {
    string message = 1;
    string type = 2;
    Kind kind = 3;
    google.protobuf.Timestamp timestamp = 4;
    Context context = 5;
    Task task = 6;
    repeated Source sources = 7;
    Target target = 8;
}

enum Kind {
    SUCCESS = 0;
    WARNING = 1;
    ERROR = 2;
}

message Task {
    string activity = 1;
    string processor = 2;
}

message Context {
    string tenant = 1;
    string user = 2;
}

message Source {
    string type = 1;
    map<string, string> attributes = 2;
}

message Target {
    string type = 1;
    map<string, string> attributes = 2;
}
// [END messages]

========= Prod environment details

Eclipse OpenJ9 VM AdoptOpenJDK (build openj9-0.21.0, JRE 11 Linux amd64-64-Bit Compressed References 20200715_697 (JIT enabled, AOT enabled)

OpenJ9   - 34cf4c075

OMR      - 113e54219

JCL      - 95bb504fbb based on jdk-11.0.8+10)

Protobuf 3.8.0

[Derek Perez]

You may want to update to protobuf-java 3.17 at least and see if it's still an issue. 

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/protobuf/e1f3c3cc-6a86-46a9-81d3-b7d002065aeen%40googlegroups.com.

[Jamu]

Thanks Derek for your quick answer!

Do you have a reason in mind for trying protobuf-java >= 3.17 ? I mean, was there any important changes around this release that might be related to this issue ?

I will probably have to fight very seriously to push for an upgrade of the version we are currently using in production, so if you have any pointers to a changelog / commit, that would be awesome!

[Derek Perez]

I definitely know we've addressed some issues around parseFrom throwing things that aren't InvalidProtocolBufferException. Although, without any reproduction case, it's really hard to say what's up.

To view this discussion on the web visit https://groups.google.com/d/msgid/protobuf/afaed1a6-d3ef-4c21-a33e-091030cb24den%40googlegroups.com.