DefaultIdStrategy group filter recursion - for deep clone

[Bogdan Vasile]

Hi,

I'm trying to use Protostuff to deep clone object graphs. Recently I encountered a situation where I needed to clone only a subset of the entities fields (a subgraph of an entity).

I used the example from io.protostuff.runtime.FilterFieldsByGroupTest. I annotated all the fields with the correct group filter id, but it looks like the fields filtering work only for the base entity, not its sub-entities.

E.g.:

For the following entities:

public class Ent_Network{

        public static final int NETWORK_CLONING_GROUP = 1;

        

        @Tag(1)

private String code;

        @Tag(value = 2, groupFilter = Ent_Network.NETWORK_CLONING_GROUP)

private Ent_NetworkType networkType;

        @Tag(value = 3, groupFilter = Ent_Network.NETWORK_CLONING_GROUP)

private Set<Ent_NetworkLocation> networkLocations;

}

public class Ent_NetworkLocation {

        @Tag(value = 1, groupFilter = Ent_Network.NETWORK_CLONING_GROUP)

private Ent_Network network;

        @Tag(value = 2, groupFilter = Ent_Network.NETWORK_CLONING_GROUP)

private Ent_Location location;

}

public class Ent_Location {

        public static final int LOCATION_CLONING_GROUP = 2;

        @Tag(1)

private String code;

        @Tag(value = 2, groupFilter = Ent_Location.LOCATION_CLONING_GROUP)

private Set<Ent_NetworkLocation> networkLocations;

        @Tag(value = 3, groupFilter = Ent_Location.LOCATION_CLONING_GROUP)

private Set<SomeOtherEntity> someOtherEntitiesClonableOnlyFromLocation;

}

If I run the following test:

private static void testSerializationFilter(Ent_Network network){

DefaultIdStrategy primary = new DefaultIdStrategy();

        DefaultIdStrategy g1 = new DefaultIdStrategy(primary, Ent_Network.NETWORK_CLONING_GROUP);

        Schema<Ent_Network> schema = RuntimeSchema.getSchema(Ent_Network.class, g1);

        byte[] data = ProtostuffIOUtil.toByteArray(network, schema, LinkedBuffer.allocate(256));

        Ent_Network receivedNetwork = schema.newMessage();

        ProtostuffIOUtil.mergeFrom(data, receivedNetwork, schema);

}

I would expect the fields networkLocations and someOtherEntitiesClonableOnlyFromLocation from Locations from the network locations to be null. Instead, they are populated. I think this happens because the groupId is not handed over to the schemas created for the subentities.

Do you have any idea on how to fix this?

[David Yu]

Correct. 

Instead, they are populated. I think this happens because the groupId is not handed over to the schemas created for the subentities.

Do you have any idea on how to fix this?

I've fixed this in my local repo.  I'll add some more tests and then I shall push the fix.

--
You received this message because you are subscribed to the Google Groups "protostuff" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protostuff+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

When the cat is away, the mouse is alone.
dyuproject.com

[Bogdan Vasile]

David, thank you for your response.

 I tried fixing this issue myself, but I found that the root of the problem is caused by the primaryGroup that exists in the  IdStrategy.

 

The schema for the root entity is generated with the correct group and the overloaded strategy, but for nested entities the primaryGroup (IdStrategy) is used (passed as a parameter), which has a groupId of 0.

 

The workaround that worked for me was to pass a null primaryGroup when creating IdStrategies and allow a non-zero groupId in the IdStrategy constructor. This way I made sure that all the generated schemas would use the correct  groupId. But that means that for each groupId I'll have to store a separated set of strategies, causing a larger memory footprint.

Anyway, I'll look on github for your modifications and tell you how it works for me.

[David Yu]

On Mon, Nov 6, 2017 at 7:41 PM, Bogdan Vasile <bogdan...@gmail.com> wrote:

David, thank you for your response.

 I tried fixing this issue myself, but I found that the root of the problem is caused by the primaryGroup that exists in the  IdStrategy.

 

The schema for the root entity is generated with the correct group and the overloaded strategy, but for nested entities the primaryGroup (IdStrategy) is used (passed as a parameter), which has a groupId of 0.

 

The workaround that worked for me was to pass a null primaryGroup when creating IdStrategies and allow a non-zero groupId in the IdStrategy constructor. This way I made sure that all the generated schemas would use the correct  groupId. But that means that for each groupId I'll have to store a separated set of strategies, causing a larger memory footprint.

Right.

Good job identifying the problem and having a workaround for it.

My fix keeps the current semantics and touches a few files.

Anyway, I'll look on github for your modifications and tell you how it works for me.

Ok.  Btw, are you deploying protostuff in production?  Do you have a link? 

--
You received this message because you are subscribed to the Google Groups "protostuff" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protostuff+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Bogdan Vasile]

I'm using Protostuff on a project that is still in development, which should soon be production ready. But all the modifications that I've done are on my local HDD, and I'm not using git, nor github.

Do you want me to send you a diff file with my changes?

[David Yu]

On Mon, Nov 6, 2017 at 10:35 PM, Bogdan Vasile <bogdan...@gmail.com> wrote:

I'm using Protostuff on a project that is still in development, which should soon be production ready. But all the modifications that I've done are on my local HDD, and I'm not using git, nor github.

Ok. Btw, I meant links to your product/project, not the git repo.

Do you want me to send you a diff file with my changes?

No.  It's not necessary.  Thanks

--
You received this message because you are subscribed to the Google Groups "protostuff" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protostuff+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Bogdan Vasile]

Well, the project that we developped was a custom project for the Romanian Environment Ministry, so we don't have a product link. But you can access the result at www.calitateaer.ro . The internal communication between the backends and frontends uses protostuff.

[David Yu]

On Mon, Nov 6, 2017 at 11:04 PM, Bogdan Vasile <bogdan...@gmail.com> wrote:

Well, the project that we developped was a custom project for the Romanian Environment Ministry, so we don't have a product link. But you can access the result at www.calitateaer.ro . The internal communication between the backends and frontends uses protostuff.

Thanks for the link.  So you're using it for rpc.  I'm assuming the data passed between those two are cyclic? (your examples were tailored for protostuff's graph format)

--
You received this message because you are subscribed to the Google Groups "protostuff" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protostuff+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Bogdan Vasile]

Yes, we are using it for RPC. Most entities that we pass around are cyclic, and to simplify the communication, all the serializations and deserializations are done using the GraphIOUtil.