BSONObjectID java serialization is not backwards compatible from 0.12.5 to 0.12.7

[João Ferreira]

Hi

We found that the bytecode fields of reactivemongo.bson.BSONObjectID changed from 0.12.5 to 0.12.7. This is not a request for any change just a warning for users that might be using java serialization of BSONObjectID. 

I extracted the class file from the jars and ran javap against them (emphasis mine)

~/.i/c/o/r/jars $ javap -p BSONObjectID-12.5.class 

Compiled from "types.scala"

public class reactivemongo.bson.BSONObjectID implements reactivemongo.bson.BSONValue,scala.Serializable,scala.Equals {

  public static final long serialVersionUID;

  private final byte[] raw;

  private final byte code;

  private java.lang.String stringify;

  private int hashCode;

  private volatile byte bitmap$0;

  public static boolean fromTime$default$2();

  public static reactivemongo.bson.BSONObjectID fromTime(long, boolean);

  public static reactivemongo.bson.BSONObjectID generate();

  public static scala.util.Try<reactivemongo.bson.BSONObjectID> parse(java.lang.String);

  public static scala.Option<byte[]> unapply(reactivemongo.bson.BSONObjectID);

  public static reactivemongo.bson.BSONObjectID apply(byte[]);

  public static reactivemongo.bson.BSONObjectID apply(java.lang.String);

  private java.lang.String stringify$lzycompute();

  private int hashCode$lzycompute();

  private byte[] raw();

  public byte code();

  public java.lang.String stringify();

  public java.lang.String toString();

  public boolean canEqual(java.lang.Object);

  public boolean equals(java.lang.Object);

  public int hashCode();

  public long time();

  public int timeSecond();

  public byte[] valueAsArray();

  public reactivemongo.bson.BSONObjectID(byte[]);

}

~/.i/c/o/r/jars $ javap -p BSONObjectID-12.7.class 

Compiled from "types.scala"

public class reactivemongo.bson.BSONObjectID implements reactivemongo.bson.BSONValue,scala.Serializable,scala.Equals {

  public static final long serialVersionUID;

  private final byte[] reactivemongo$bson$BSONObjectID$$raw;

  private final byte code;

  private java.lang.String stringify;

  private int hashCode;

  private volatile byte bitmap$0;

  public static boolean fromTime$default$2();

  public static reactivemongo.bson.BSONObjectID fromTime(long, boolean);

  public static reactivemongo.bson.BSONObjectID generate();

  public static scala.util.Try<reactivemongo.bson.BSONObjectID> parse(java.lang.String);

  public static scala.Option<byte[]> unapply(reactivemongo.bson.BSONObjectID);

  public static reactivemongo.bson.BSONObjectID apply(byte[]);

  public static reactivemongo.bson.BSONObjectID apply(java.lang.String);

  private java.lang.String stringify$lzycompute();

  private int hashCode$lzycompute();

  public byte[] reactivemongo$bson$BSONObjectID$$raw();

  public byte code();

  public java.lang.String stringify();

  public java.lang.String toString();

  public boolean canEqual(java.lang.Object);

  public boolean equals(java.lang.Object);

  public int hashCode();

  public long time();

  public int timeSecond();

  public byte[] valueAsArray();

  public int byteSize();

  public reactivemongo.bson.BSONObjectID(byte[]);

}

And here is the diff of the above output

~/.i/c/o/r/jars $ diff (javap -p BSONObjectID-12.5.class|psub) (javap -p BSONObjectID-12.7.class|psub)

4c4

<   private final byte[] raw;

---

>   private final byte[] reactivemongo$bson$BSONObjectID$$raw;

18c18

<   private byte[] raw();

---

>   public byte[] reactivemongo$bson$BSONObjectID$$raw();

27a28

>   public int byteSize();

~/.i/c/o/r/jars $ 

The only explanation I can found for this is the commit https://github.com/ReactiveMongo/ReactiveMongo/commit/65ea55170aedf29660cb4e1aff312ede30e3414d#diff-889a53eab1347791b8e8e140dc7a1b77R395 that introduced:

@inline override private[reactivemongo] def byteSize = raw.size

Since this method is annotated with @inline and it references the private field raw, the scala compiler must have decided to compile things differently.

Cheers

[Cédric Chantepie]

These version are API compatibles, on the other side the mentioned field is new and private so cannot be referrenced by external code, so unless a specific issue is found, that's "just" details of internal implementations.

[João Ferreira]

You are right that the versions are API compatible (and your use of MiMa guarantees that), but they are not compatible in the java serialization sense[1] because the field "raw" as renamed (by the compiler) to "reactivemongo$bson$BSONObjectID$$raw" 

The specific issue that I face was that I had a BSONObjectId saved in a cache, and when I rolled the reactivemongo upgrade to PROD, the BSONObjectId was successfully deserialized but I would get an NullPointerException as soon the "raw" field was accessed. I fixed the issue by flushing the cache. For reference here is the stacktrace:

Oct 17 22:26:26 app/web.1: java.lang.NullPointerException: null

Oct 17 22:26:26 app/web.1:     at reactivemongo.bson.utils.Converters$.hex2Str(utils.scala:24)

Oct 17 22:26:26 app/web.1:     at reactivemongo.bson.BSONObjectID.stringify$lzycompute(types.scala:374)

Oct 17 22:26:26 app/web.1:     at reactivemongo.bson.BSONObjectID.stringify(types.scala:374)

Oct 17 22:26:26 app/web.1:     at reactivemongo.bson.BSONObjectID.toString(types.scala:376)

One can say that java serialization should not be used for anything important (and I agree), but for those that made the same mistake I did be aware of this issue.

Cheers

João Ferreira

[1] http://docs.oracle.com/javase/6/docs/platform/serialization/spec/version.html#6519

2017-10-18 12:50 GMT+01:00 Cédric Chantepie <chantep...@gmail.com>:

These version are API compatibles, on the other side the mentioned field is new and private so cannot be referrenced by external code, so unless a specific issue is found, that's "just" details of internal implementations.

--
You received this message because you are subscribed to a topic in the Google Groups "ReactiveMongo - http://reactivemongo.org" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/reactivemongo/dYmmPSpSUyg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to reactivemongo+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.