How to combine two Different Protobuf Messages

[akshay padmanabhan]

Hello 

I'm pretty new to protobuf and kinda stuck with this problem and want some ideas from people who use this extensively.

I have two message 

Message A {

      string foo = 1;

      string bar = 2;

}

Message B {

     string hello = 1;

     string world = 2;

}

Now I have 2 sources a SQL and another Hive. And the SQL source actually returns A. So for SQL it is READ<A> and B which is a meta data we get this from SQL queries. So we never needed B all this while as A did the job for us.

Now our pipelines will start reading from the Hive as well. But for hive, we have these Metadata as columns of record .. So now I have (A + B) combined.

Since the code was initially written for SQL the READ<A> Facade can't be modified as client code will change.

but inside I need to now do some merging of A and B dynamically before actually calling Hive.

Is this possible in protobuf? I checked if Dynamic message can help me. I couldn't get it to resolve my issue. Can't use mergeFrom as well as both the message types need to be equal.

I can't do 

Message C {

Message A 

Message B

}

This will change the eventual results of the record having extra columns.

I need something like mergeFrom for 2 different messages type. I have access to both the proto's inside the READ. So I can do anything and everything with these two proto's. But eventually, I want a merged proto C from A and B by code and not by .proto