Empty ArrayList data member being deserialized back as null

[Wilson]

Hi,

We are seeing when a class that contains an empty ArrayList is being serialized and then deserialized back as null. Below are code snippet to illustrate this:

package com.test;

import java.util.ArrayList;

import java.util.List;

import com.dyuproject.protostuff.LinkedBuffer;

import com.dyuproject.protostuff.ProtobufIOUtil;

import com.dyuproject.protostuff.Schema;

import com.dyuproject.protostuff.runtime.RuntimeSchema;

public class ProtostuffEmptyArrayListTest {

private List<String> listVal;

public List<String> getListVal() {

return listVal;

}

public void setListVal(List<String> listVal) {

this.listVal = listVal;

}

public static void main(String[] args) {

ProtostuffEmptyArrayListTest test = new ProtostuffEmptyArrayListTest();

test.setListVal(new ArrayList<String>());

Schema<ProtostuffEmptyArrayListTest> schema = 

RuntimeSchema.getSchema(ProtostuffEmptyArrayListTest.class);

// Serialize

LinkedBuffer buff = LinkedBuffer.allocate(LinkedBuffer.DEFAULT_BUFFER_SIZE);

byte[] serailzedContent = ProtobufIOUtil.toByteArray(test, schema, buff);

// Deserialize

ProtostuffEmptyArrayListTest result = new ProtostuffEmptyArrayListTest();

ProtobufIOUtil.mergeFrom(serailzedContent, result, schema);

System.out.println("BeforeDeserializedContent: " + test.getListVal());

System.out.println("DeserializedContent: " + result.getListVal());

}

}

When this class is run, the result prints:

BeforeDeserializedContent: []

DeserializedContent: null

Can the result be returned back as empty list instead? It is problematic particularly when the class we are using isn't a pojo we can modified.

Thanks

Wilson.

[Wilson]

Hi,

This sample example below better illustrates our problem.

package com.test;

import java.io.ByteArrayInputStream;

import java.io.ByteArrayOutputStream;

import java.io.ObjectInputStream;

import java.io.ObjectOutputStream;

import java.io.Serializable;

import java.util.ArrayList;

import java.util.Collection;

import com.dyuproject.protostuff.LinkedBuffer;

import com.dyuproject.protostuff.ProtobufIOUtil;

import com.dyuproject.protostuff.Schema;

import com.dyuproject.protostuff.runtime.RuntimeSchema;

class TestClass1 implements Serializable {

private TestClass2 testObj;

public TestClass2 getTestObj() {

return testObj;

}

public void setTestObj(TestClass2 testObj) {

this.testObj = testObj;

}

}

class TestClass2 implements Serializable {

private Collection<String> collVal = new ArrayList<String>();

private String testStr;

public TestClass2(String testStr) {

this.testStr = testStr;

}

public Collection<String> getCollVal() {

return collVal;

}

public void setCollVal(Collection<String> collVal) {

this.collVal = collVal;

}

public String getTestStr() {

return testStr;

}

public void setTestStr(String testStr) {

this.testStr = testStr;

}

}

public class ProtostuffEmptyArrayListTest {

public static void main(String[] args) {

try {

TestClass1 test = new TestClass1();

test.setTestObj(new TestClass2("testVal"));

Schema<TestClass1> schema = RuntimeSchema.getSchema(TestClass1.class);

// Serialize with Protostuff serialization

LinkedBuffer buff = LinkedBuffer.allocate(LinkedBuffer.DEFAULT_BUFFER_SIZE);

byte[] protostuffSerailzedContent = ProtobufIOUtil.toByteArray(test, schema, buff);

// Deserialize with Protostuff serialization

TestClass1 result1 = new TestClass1();

ProtobufIOUtil.mergeFrom(protostuffSerailzedContent, result1, schema);

// Serialize with Java serialzation.

ByteArrayOutputStream bos = new ByteArrayOutputStream();

ObjectOutputStream out = new ObjectOutputStream(bos);

out.writeObject(test);

out.close();

byte[] javaSerialzedContent = bos.toByteArray();

// Deserialize with Java serialization.

ObjectInputStream in = new ObjectInputStream(new ByteArrayInputStream(javaSerialzedContent));

TestClass1 result2 = (TestClass1)in.readObject();

System.out.println("BeforeDeserializedContent: " + test.getTestObj().getCollVal());

System.out.println("ProtostuffSerialization - DeserializedContent: " + result1.getTestObj().getCollVal());

System.out.println("JavaSerialization - DeserializedContent: " + result2.getTestObj().getCollVal());

} catch (Exception ex) {

System.err.println("Error: " + ex);

}

}

}

The output result from running the above code is:

BeforeDeserializedContent: []

ProtostuffSerialization - DeserializedContent: null

JavaSerialization - DeserializedContent: []

So even though TestClass2 initializes the collVal to an empty ArrayList, the protostuff deserialized result makes this collVal as null. It seems to happen when we have a non-default constructor defined in TestClass2 and no default constructor defined.

Just wanted to be sure this wasn't a behaviour in Java by default, I've tried to perform this with Java Serialization, and the deserialized result seem to be ok with Java Serialization.

Are you able to help?

Thanks

Regards

Wilson.

[David Yu]

By default, protostuff doesn't serialize the collection itself, only their values.

The simplest way you can solve that is annotating the List<String> listVal with @Morph.

That tells protostuff to serialize the collection itself.  

The downside to that, extra metadata is included because the type is a interface (List)

To avoid extra metadata when annotated with @Morph , use the concrete type.  E.g. ArrayList<String> listVal.

If you need it to be a list and cannot annotate it with @Morph (third-party code), set the property:

-Dprotostuff.runtime.morph_collection_interfaces=true

 

Thanks

Wilson.

--
When the cat is away, the mouse is alone.
- David Yu

[David Yu]

What's happening there is that TestClass2 doesn't have a no-arg constructor.

When that happens, sun's reflection factory is used and the object's fields that need to be initialized on construction, will not be initialized.

Simplest solution is to annotate with @Morph.

Otherwise, you'll have to add a no-arg constructor so that those fields will be initialized.

[Wilson]

Hi David,

I've not made any change, but just added "-Dprotostuff.runtime.collection_schema_on_repeated_fields=true" (as I needed to do that for using with GraphIOUtil to handle cylic collection fields on another object graph pojo). This issue seem to have gone by adding the above -D instead of adding "-Dprotostuff.runtime.morph_collection_interfaces=true". Is that what you would have expected?

Thanks

Regards

Wilson.

On Tuesday, July 3, 2012 11:21:57 AM UTC+1, Wilson wrote:

[David Yu]

On Tue, Jul 10, 2012 at 4:32 AM, Wilson <wilson...@hotmail.com> wrote:

Hi David,

I've not made any change, but just added "-Dprotostuff.runtime.collection_schema_on_repeated_fields=true" (as I needed to do that for using with GraphIOUtil to handle cylic collection fields on another object graph pojo). This issue seem to have gone by adding the above -D instead of adding "-Dprotostuff.runtime.morph_collection_interfaces=true". Is that what you would have expected?

Yes.  If the same collection was referenced by another pojo, only  the reference id will be written.