How to migrate to protocol buffers from bean style data structure

[atulat...@gmail.com]

Hi,
I have couple of questions regarding protocol buffers in java
1. I am using old bean style data structures. How can i migrate to
protocol buffers and what problems may arise due to this migration.
2. How can i serialize bean style data structures using protocol
buffers?

[Kenton Varda]

On Tue, Dec 16, 2008 at 2:27 AM, <atulat...@gmail.com> wrote:

Hi,
I have couple of questions regarding protocol buffers in java
1. I am using old bean style data structures. How can i migrate to
protocol buffers and what problems may arise due to this migration.

This is a pretty broad question with many reasonable answers.  You could keep your existing structures and just add methods to convert between them and protocol buffers (see below).  This would be easy but would be annoying to maintain since every time you added a field you'd have to update the bean class.  Alternatively, you could completely replace your bean-style classes with protocol message classes, but this will require refactoring your code to work with immutable objects.  Most code tends to be pretty easy to refactor (because most code will tend to construct data objects all at once and then not modify them afterwards, which is what protocol buffer objects require), but some code may be more complicated to rework.  Also, if you have a lot of code it could be painful.  A third option is to replace your bean objects with protocol message builders and use those everywhere, since they have both setters and getters and thus can operate like mutable objects.  However, this is kind of ugly and not the way builders are meant to be used.

I would probably recommend the first option, as it keeps your code more independent of protocol buffers, so you can more easily change your mind later if you want to.

 

2. How can i serialize bean style data structures using protocol
buffers?

You'd need to add methods which convert between your bean class and protocol message objects.  For example:

  public MyMessageType toProto() {

    return MyMessageType.newBuilder()

      .setFoo(this.getFoo())

      .setBar(this.getBar())

      .setBaz(this.getBaz())

      .build();

  }

  public void fromProto(MyMessageType proto) {

    this.setFoo(proto.getFoo());

    this.setBar(proto.getBar());

    this.setBaz(proto.getBaz());

  }

[atulat...@gmail.com]

Thanks for answer Ketan.

On Dec 17, 12:41 am, Kenton Varda <ken...@google.com> wrote:

> On Tue, Dec 16, 2008 at 2:27 AM, <atulatri2...@gmail.com> wrote:
>
> > Hi,
> > I have couple of questions regarding protocol buffers in java
> > 1. I am using old bean style data structures. How can i migrate to
> > protocol buffers and what problems may arise due to this migration.
>
> This is a pretty broad question with many reasonable answers.  You could
> keep your existing structures and just add methods to convert between them
> and protocol buffers (see below).  This would be easy but would be annoying
> to maintain since every time you added a field you'd have to update the bean
> class.  Alternatively, you could completely replace your bean-style classes
> with protocol message classes, but this will require refactoring your code
> to work with immutable objects.  Most code tends to be pretty easy to
> refactor (because most code will tend to construct data objects all at once
> and then not modify them afterwards, which is what protocol buffer objects
> require), but some code may be more complicated to rework.  Also, if you
> have a lot of code it could be painful.  A third option is to replace your
> bean objects with protocol message builders and use those everywhere, since
> they have both setters and getters and thus can operate like mutable
> objects.  However, this is kind of ugly and not the way builders are meant
> to be used.
>
> I would probably recommend the first option, as it keeps your code more
> independent of protocol buffers, so you can more easily change your mind
> later if you want to.

In my case things are little different. Bean's object data is set from
network resources and some are from program itself. And then these
objects are used every where in program. So idea of keeping, both
codes of bean and protocol buffer separate and introduce methods in
bean classes, seems will not work unless we refactor code.
Yes another idea is to replace all the bean styled data structures
with proto buffers is ultimate and will work, But since code is too
much and this replacement will surely give pain, We keep this option
last.
Another way to resolve this problem is to make a interface with getter
methods and both bean styled classes and protobuff message are
refactored to implement this interface and then use this interface
everywhere. This also require refactoring.
It seems this problem can not be solved with out refactoring the code.

>
> > 2. How can i serialize bean style data structures using protocol
> > buffers?
>
> You'd need to add methods which convert between your bean class and protocol
> message objects.  For example:
>
>   public MyMessageType toProto() {
>     return MyMessageType.newBuilder()
>       .setFoo(this.getFoo())
>       .setBar(this.getBar())
>       .setBaz(this.getBaz())
>       .build();
>   }
>
>   public void fromProto(MyMessageType proto) {
>     this.setFoo(proto.getFoo());
>     this.setBar(proto.getBar());
>     this.setBaz(proto.getBaz());
>   }

This way is interesting. But require protocol buffers data structure
for all bean classes those i want to serialize. And when i want to
serialize bean class just convert it to protocol buffer message type
calling toProto() method and then serialize it.
But what if i have a list of bean styled data structure objects mixed
with protocol buffers objects and want to serialize it?? A problem
area?
Now to serialize i can use XStream that will serialize such list of
bean styled data structure objects mixed with protocol buffers objects
but problem is size of file of serialized objects that XStream will
generate. This file size would be many fold bigger that protocol
buffers is able to generate.
I would like to to ask if protocol buffer is going to have such
XStream like feature where we can serialize object specially value
objects without making any changes.
Any other idea on serialization of objects that can harness the power
of protocol buffers without making any changes in value object classes
like XStream .

[Kenton Varda]

You might be interested in the protobuf reflection interface.  You can use Descriptors to examine a message type and then use the methods of the Message and Builder interfaces (e.g. Message.getField()) to examine the fields of an actual message object.  With these, you could write a generic algorithm which converts between arbitrary bean objects and protocol buffers.

http://code.google.com/apis/protocolbuffers/docs/reference/java/com/google/protobuf/Descriptors.html

http://code.google.com/apis/protocolbuffers/docs/reference/java/com/google/protobuf/Message.html

http://code.google.com/apis/protocolbuffers/docs/reference/java/com/google/protobuf/Message.Builder.html