Serializing sub classes that extend Super class...
932 views
Skip to first unread message
Robert Mann
Feb 15, 2012, 4:51:04 PM2/15/12
to kryonet-users
I have a super class Card that has hundreds of subclasses that extend
it.
I keep all of the Card objects (including its subclasses) in an
ArrayList<Card>. I want to send the ArrayList of Card objects using
Kryo net. Must I register every single one of the Card subclasses?
Or is there a way to just register the Card object and have it still
able to send the subclasses?
it.
I keep all of the Card objects (including its subclasses) in an
ArrayList<Card>. I want to send the ArrayList of Card objects using
Kryo net. Must I register every single one of the Card subclasses?
Or is there a way to just register the Card object and have it still
able to send the subclasses?
Sam Ritchie
Feb 27, 2012, 4:45:18 PM2/27/12
to kryone...@googlegroups.com, Oscar Boykin
I ran into a similar problem with Kryo, looking to register a serializer for Thrift objects. They're all serialized the same way, but Kryo doesn't provide a way to register superclasses. Oscar Boykin and I solved this in Cascading.Kryo by extending Kryo to know about "hierarchies":
--
If you guys find this useful, we can push a patch to Kryo proper.
Cheers,
Sam
--
You received this message because you are subscribed to the "kryonet-users" group.
http://groups.google.com/group/kryonet-users
Sam Ritchie, Twitter Inc
@sritchie09
(Too brief? Here's why! http://emailcharter.org)
Ax
Mar 5, 2012, 5:38:01 PM3/5/12
to kryonet-users
I believe you do have to register them all, including ArrayList.
Robert Mann
Mar 6, 2012, 9:37:53 PM3/6/12
to kryonet-users
Ok sounds good. But I am not quite sure how to use your Kryo. Can
you give an example?
I have a network class in which I do the serializing that has a
register method that looks like:
public static void register(EndPoint endPoint)
{
Kryo kryo = endPoint.getKryo();
kryo.register....
}
How can I get an instance of your Kryo class so that I can call:
kryo.registerHierarchy( some.class, serialize);
On Feb 27, 4:45 pm, Sam Ritchie <sritchi...@gmail.com> wrote:
> I ran into a similar problem with Kryo, looking to register a serializer
> for Thrift objects. They're all serialized the same way, but Kryo doesn't
> provide a way to register superclasses. Oscar Boykin and I solved this in
> Cascading.Kryo by extending Kryo to know about "hierarchies":
>
> https://github.com/Cascading/cascading.kryo/blob/master/src/jvm/casca...
you give an example?
I have a network class in which I do the serializing that has a
register method that looks like:
public static void register(EndPoint endPoint)
{
Kryo kryo = endPoint.getKryo();
kryo.register....
}
How can I get an instance of your Kryo class so that I can call:
kryo.registerHierarchy( some.class, serialize);
On Feb 27, 4:45 pm, Sam Ritchie <sritchi...@gmail.com> wrote:
> I ran into a similar problem with Kryo, looking to register a serializer
> for Thrift objects. They're all serialized the same way, but Kryo doesn't
> provide a way to register superclasses. Oscar Boykin and I solved this in
> Cascading.Kryo by extending Kryo to know about "hierarchies":
>
>
> If you guys find this useful, we can push a patch to Kryo proper.
>
> Cheers,
> Sam
>
> If you guys find this useful, we can push a patch to Kryo proper.
>
> Cheers,
> Sam
>
Nate
Mar 6, 2012, 9:45:52 PM3/6/12
to kryone...@googlegroups.com
Yes, you have to register all your classes. This gives them an int ID, which is used in the serialized bytes to identify the class. If you use Kryo#setRegistrationOptional(true), then classes that aren't registered can be used. Their class name will be embedded the first time they are encountered in an object graph, so will use more bytes.
Depending on your object model, you could refactor your card class to have an enum to identify the card type rather than subclasses.
-Nate
Depending on your object model, you could refactor your card class to have an enum to identify the card type rather than subclasses.
-Nate
Sam Ritchie
Mar 8, 2012, 4:03:19 AM3/8/12
to kryone...@googlegroups.com
Robert, I think you'd have to include my project as a dependency and explicitly instantiate it with
Kryo kryo = new cascading.kryo.Kryo();
If you're retrieving an instance of Kryo from an Endpoint, I'm not sure you can use this without redoing the endpoint to return an instance of cascading.kryo.Kryo.
Cheers,
Robert Mann
Mar 8, 2012, 10:46:09 PM3/8/12
to kryonet-users
I'm not sure how to tie:
I thought about changing Endpoint to return the cascading.kryo but I
have a feeling that's just going to ripple in to other things.
What I've ended up doing is created a TransportCard.class have all the
cards be converted into before sending it. I know this sounds really
stupid, but since the game is a "Dumb Client/Smart Server" model I can
cut a lot of the extraneous information out of the TransportCards. It
ends up reducing bandwidth usage for these objects by about 65%.
There are a few places where I need to actually send the unique Card
object, and for those cases I'm just using Nate's (thank you)
suggestion above: kryo.setRegistrationOptional(true); It's not ideal
since the serialization is inefficient, but it's not a significant
bandwidth use.
I still would like to use the cascading.kryo if I can just figure out
how to tie it to my Client object.
On Mar 8, 4:03 am, Sam Ritchie <sritchi...@gmail.com> wrote:
> Robert, I think you'd have to include my project as a dependency and explicitly instantiate it with
>
> Kryo kryo = new cascading.kryo.Kryo();
>
> If you're retrieving an instance of Kryo from an Endpoint, I'm not sure you can use this without redoing the endpoint to return an instance of cascading.kryo.Kryo.
>
> Cheers,
> --
> Sam Ritchie
> Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
Kryo kryo = new cascading.kryo.Kryo();
To my Client object.
I thought about changing Endpoint to return the cascading.kryo but I
have a feeling that's just going to ripple in to other things.
What I've ended up doing is created a TransportCard.class have all the
cards be converted into before sending it. I know this sounds really
stupid, but since the game is a "Dumb Client/Smart Server" model I can
cut a lot of the extraneous information out of the TransportCards. It
ends up reducing bandwidth usage for these objects by about 65%.
There are a few places where I need to actually send the unique Card
object, and for those cases I'm just using Nate's (thank you)
suggestion above: kryo.setRegistrationOptional(true); It's not ideal
since the serialization is inefficient, but it's not a significant
bandwidth use.
I still would like to use the cascading.kryo if I can just figure out
how to tie it to my Client object.
On Mar 8, 4:03 am, Sam Ritchie <sritchi...@gmail.com> wrote:
> Robert, I think you'd have to include my project as a dependency and explicitly instantiate it with
>
> Kryo kryo = new cascading.kryo.Kryo();
>
> If you're retrieving an instance of Kryo from an Endpoint, I'm not sure you can use this without redoing the endpoint to return an instance of cascading.kryo.Kryo.
>
> Cheers,
> --
> Sam Ritchie
>
>
>
>
>
>
>
> On Tuesday, March 6, 2012 at 6:37 PM, Robert Mann wrote:
> > Ok sounds good. But I am not quite sure how to use your Kryo. Can
> > you give an example?
>
> > I have a network class in which I do the serializing that has a
> > register method that looks like:
>
> > public static void register(EndPoint endPoint)
> > {
>
> > Kryo kryo = endPoint.getKryo();
> > kryo.register....
>
> > }
>
> > How can I get an instance of your Kryo class so that I can call:
> > kryo.registerHierarchy( some.class, serialize);
>
>
>
>
>
>
>
> On Tuesday, March 6, 2012 at 6:37 PM, Robert Mann wrote:
> > Ok sounds good. But I am not quite sure how to use your Kryo. Can
> > you give an example?
>
> > I have a network class in which I do the serializing that has a
> > register method that looks like:
>
> > public static void register(EndPoint endPoint)
> > {
>
> > Kryo kryo = endPoint.getKryo();
> > kryo.register....
>
> > }
>
> > How can I get an instance of your Kryo class so that I can call:
> > kryo.registerHierarchy( some.class, serialize);
>
> > On Feb 27, 4:45 pm, Sam Ritchie <sritchi...@gmail.com (http://gmail.com)> wrote:
> > > I ran into a similar problem with Kryo, looking to register a serializer
> > > for Thrift objects. They're all serialized the same way, but Kryo doesn't
> > > provide a way to register superclasses. Oscar Boykin and I solved this in
> > > Cascading.Kryo by extending Kryo to know about "hierarchies":
>
> > >https://github.com/Cascading/cascading.kryo/blob/master/src/jvm/casca...
>
> > > If you guys find this useful, we can push a patch to Kryo proper.
>
> > > Cheers,
> > > Sam
>
> > > I ran into a similar problem with Kryo, looking to register a serializer
> > > for Thrift objects. They're all serialized the same way, but Kryo doesn't
> > > provide a way to register superclasses. Oscar Boykin and I solved this in
> > > Cascading.Kryo by extending Kryo to know about "hierarchies":
>
> > >https://github.com/Cascading/cascading.kryo/blob/master/src/jvm/casca...
>
> > > If you guys find this useful, we can push a patch to Kryo proper.
>
> > > Cheers,
> > > Sam
>
Sam Ritchie
Mar 9, 2012, 1:56:43 AM3/9/12
to kryone...@googlegroups.com
Nate,
Would you accept a patch into Kryo that implements this hierarchy behavior? It really just serves to make the final serializer check more extensible. (Right now, if no serializer's found you default lists to ListSerializer, maps to MapSerializer, and all else to FieldsSerializer. the hierarchies patch would allow you to insert other checks in advance of this, so if some class is assignable from a given interface or superclass, it would shunt off to that registered "hierarchy serializer".
Thanks!
Sam
Nate
Mar 9, 2012, 4:07:46 AM3/9/12
to kryone...@googlegroups.com
I believe you can achieve that by extending Kryo and overriding "Serializer newDefaultSerializer (Class type)". In there, do instanceof checks and return the appropriate serializer. You still need to register the individual classes though, because each one needs a int ID (unless using optional registration of course).
-Nate
-Nate
Sam Ritchie
Mar 9, 2012, 10:40:31 AM3/9/12
to kryone...@googlegroups.com
Yep, that's what we did in cascading.kryo:
What are your thoughts on including this feature in Kryo proper?
Thanks so much for this library, by the way! It's been a real blessing to Cascalog, and is in use in production here at Twitter.
Cheers,
Sam
Robert Mann
Mar 10, 2012, 9:34:37 PM3/10/12
to kryonet-users
I decided to just add your cascading.kryo code directly into the
existing kryo class (because I'm not sure how to create the
cascading.kryo from Endpoint). Works great, thank you so much! It
decreased 6000 lines of code (and an upkeep nightmare) into 300 lines
of code.
On Mar 9, 10:40 am, Sam Ritchie <sritchi...@gmail.com> wrote:
> Yep, that's what we did in cascading.kryo:
>
> https://github.com/Cascading/cascading.kryo/blob/master/src/jvm/casca...
existing kryo class (because I'm not sure how to create the
cascading.kryo from Endpoint). Works great, thank you so much! It
decreased 6000 lines of code (and an upkeep nightmare) into 300 lines
of code.
On Mar 9, 10:40 am, Sam Ritchie <sritchi...@gmail.com> wrote:
> Yep, that's what we did in cascading.kryo:
>
>
> What are your thoughts on including this feature in Kryo proper?
>
> Thanks so much for this library, by the way! It's been a real blessing to Cascalog, and is in use in production here at Twitter.
>
> Cheers,
> Sam
>
> --
> Sam Ritchie
> Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
> What are your thoughts on including this feature in Kryo proper?
>
> Thanks so much for this library, by the way! It's been a real blessing to Cascalog, and is in use in production here at Twitter.
>
> Cheers,
> Sam
>
> --
> Sam Ritchie
>
>
>
>
>
>
>
> On Friday, March 9, 2012 at 1:07 AM, Nate wrote:
> > I believe you can achieve that by extending Kryo and overriding "Serializer newDefaultSerializer (Class type)". In there, do instanceof checks and return the appropriate serializer. You still need to register the individual classes though, because each one needs a int ID (unless using optional registration of course).
>
> > -Nate
>
>
>
>
>
>
>
> On Friday, March 9, 2012 at 1:07 AM, Nate wrote:
> > I believe you can achieve that by extending Kryo and overriding "Serializer newDefaultSerializer (Class type)". In there, do instanceof checks and return the appropriate serializer. You still need to register the individual classes though, because each one needs a int ID (unless using optional registration of course).
>
> > -Nate
>
> > On Thu, Mar 8, 2012 at 10:56 PM, Sam Ritchie <sritchi...@gmail.com (mailto:sritchi...@gmail.com)> wrote:
> > > Nate,
>
> > > Would you accept a patch into Kryo that implements this hierarchy behavior? It really just serves to make the final serializer check more extensible. (Right now, if no serializer's found you default lists to ListSerializer, maps to MapSerializer, and all else to FieldsSerializer. the hierarchies patch would allow you to insert other checks in advance of this, so if some class is assignable from a given interface or superclass, it would shunt off to that registered "hierarchy serializer".
>
> > > Thanks!
> > > Sam
>
> > > Nate,
>
> > > Would you accept a patch into Kryo that implements this hierarchy behavior? It really just serves to make the final serializer check more extensible. (Right now, if no serializer's found you default lists to ListSerializer, maps to MapSerializer, and all else to FieldsSerializer. the hierarchies patch would allow you to insert other checks in advance of this, so if some class is assignable from a given interface or superclass, it would shunt off to that registered "hierarchy serializer".
>
> > > Thanks!
> > > Sam
>
> > > On Thu, Mar 8, 2012 at 7:46 PM, Robert Mann <thecorta...@gmail.com (mailto:thecorta...@gmail.com)> wrote:
> > > > I'm not sure how to tie:
> > > > Kryo kryo = new cascading.kryo.Kryo();
>
> > > > To my Client object.
>
> > > > I thought about changing Endpoint to return the cascading.kryo but I
> > > > have a feeling that's just going to ripple in to other things.
>
> > > > What I've ended up doing is created a TransportCard.class have all the
> > > > cards be converted into before sending it. I know this sounds really
> > > > stupid, but since the game is a "Dumb Client/Smart Server" model I can
> > > > cut a lot of the extraneous information out of the TransportCards. It
> > > > ends up reducing bandwidth usage for these objects by about 65%.
>
> > > > There are a few places where I need to actually send the unique Card
> > > > object, and for those cases I'm just using Nate's (thank you)
> > > > suggestion above: kryo.setRegistrationOptional(true); It's not ideal
> > > > since the serialization is inefficient, but it's not a significant
> > > > bandwidth use.
>
> > > > I still would like to use the cascading.kryo if I can just figure out
> > > > how to tie it to my Client object.
>
> > > > I'm not sure how to tie:
> > > > Kryo kryo = new cascading.kryo.Kryo();
>
> > > > To my Client object.
>
> > > > I thought about changing Endpoint to return the cascading.kryo but I
> > > > have a feeling that's just going to ripple in to other things.
>
> > > > What I've ended up doing is created a TransportCard.class have all the
> > > > cards be converted into before sending it. I know this sounds really
> > > > stupid, but since the game is a "Dumb Client/Smart Server" model I can
> > > > cut a lot of the extraneous information out of the TransportCards. It
> > > > ends up reducing bandwidth usage for these objects by about 65%.
>
> > > > There are a few places where I need to actually send the unique Card
> > > > object, and for those cases I'm just using Nate's (thank you)
> > > > suggestion above: kryo.setRegistrationOptional(true); It's not ideal
> > > > since the serialization is inefficient, but it's not a significant
> > > > bandwidth use.
>
> > > > I still would like to use the cascading.kryo if I can just figure out
> > > > how to tie it to my Client object.
>
> > > > On Mar 8, 4:03 am, Sam Ritchie <sritchi...@gmail.com (mailto:sritchi...@gmail.com)> wrote:
> > > > > Robert, I think you'd have to include my project as a dependency and explicitly instantiate it with
>
> > > > > Kryo kryo = new cascading.kryo.Kryo();
>
> > > > > If you're retrieving an instance of Kryo from an Endpoint, I'm not sure you can use this without redoing the endpoint to return an instance of cascading.kryo.Kryo.
>
> > > > > Cheers,
> > > > > --
> > > > > Sam Ritchie
> > > > > Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
>
> > > > > On Tuesday, March 6, 2012 at 6:37 PM, Robert Mann wrote:
> > > > > > Ok sounds good. But I am not quite sure how to use your Kryo. Can
> > > > > > you give an example?
>
> > > > > > I have a network class in which I do the serializing that has a
> > > > > > register method that looks like:
>
> > > > > > public static void register(EndPoint endPoint)
> > > > > > {
>
> > > > > > Kryo kryo = endPoint.getKryo();
> > > > > > kryo.register....
>
> > > > > > }
>
> > > > > > How can I get an instance of your Kryo class so that I can call:
> > > > > > kryo.registerHierarchy( some.class, serialize);
>
> > > > > Robert, I think you'd have to include my project as a dependency and explicitly instantiate it with
>
> > > > > Kryo kryo = new cascading.kryo.Kryo();
>
> > > > > If you're retrieving an instance of Kryo from an Endpoint, I'm not sure you can use this without redoing the endpoint to return an instance of cascading.kryo.Kryo.
>
> > > > > Cheers,
> > > > > --
> > > > > Sam Ritchie
> > > > > Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
>
> > > > > On Tuesday, March 6, 2012 at 6:37 PM, Robert Mann wrote:
> > > > > > Ok sounds good. But I am not quite sure how to use your Kryo. Can
> > > > > > you give an example?
>
> > > > > > I have a network class in which I do the serializing that has a
> > > > > > register method that looks like:
>
> > > > > > public static void register(EndPoint endPoint)
> > > > > > {
>
> > > > > > Kryo kryo = endPoint.getKryo();
> > > > > > kryo.register....
>
> > > > > > }
>
> > > > > > How can I get an instance of your Kryo class so that I can call:
> > > > > > kryo.registerHierarchy( some.class, serialize);
>
> > > > > > On Feb 27, 4:45 pm, Sam Ritchie <sritchi...@gmail.com (mailto:sritchi...@gmail.com) (http://gmail.com)> wrote:
> > > > > > > I ran into a similar problem with Kryo, looking to register a serializer
> > > > > > > for Thrift objects. They're all serialized the same way, but Kryo doesn't
> > > > > > > provide a way to register superclasses. Oscar Boykin and I solved this in
> > > > > > > Cascading.Kryo by extending Kryo to know about "hierarchies":
>
> > > > > > >https://github.com/Cascading/cascading.kryo/blob/master/src/jvm/casca...
>
> > > > > > > If you guys find this useful, we can push a patch to Kryo proper.
>
> > > > > > > Cheers,
> > > > > > > Sam
>
> > > > > > > I ran into a similar problem with Kryo, looking to register a serializer
> > > > > > > for Thrift objects. They're all serialized the same way, but Kryo doesn't
> > > > > > > provide a way to register superclasses. Oscar Boykin and I solved this in
> > > > > > > Cascading.Kryo by extending Kryo to know about "hierarchies":
>
> > > > > > >https://github.com/Cascading/cascading.kryo/blob/master/src/jvm/casca...
>
> > > > > > > If you guys find this useful, we can push a patch to Kryo proper.
>
> > > > > > > Cheers,
> > > > > > > Sam
>
> > > > > > > On Wed, Feb 15, 2012 at 1:51 PM, Robert Mann <thecorta...@gmail.com (mailto:thecorta...@gmail.com) (http://gmail.com)> wrote:
> > > > > > > > I have a super class Card that has hundreds of subclasses that extend
> > > > > > > > it.
>
> > > > > > > > I keep all of the Card objects (including its subclasses) in an
> > > > > > > > ArrayList<Card>. I want to send the ArrayList of Card objects using
> > > > > > > > Kryo net. Must I register every single one of the Card subclasses?
> > > > > > > > Or is there a way to just register the Card object and have it still
> > > > > > > > able to send the subclasses?
>
> > > > > > > > --
> > > > > > > > You received this message because you are subscribed to the
> > > > > > > > "kryonet-users" group.
> > > > > > > >http://groups.google.com/group/kryonet-users
>
> > > > > > > --
> > > > > > > Sam Ritchie, Twitter Inc
> > > > > > > 703.662.1337 (tel:703.662.1337)
> > > > > > > > I have a super class Card that has hundreds of subclasses that extend
> > > > > > > > it.
>
> > > > > > > > I keep all of the Card objects (including its subclasses) in an
> > > > > > > > ArrayList<Card>. I want to send the ArrayList of Card objects using
> > > > > > > > Kryo net. Must I register every single one of the Card subclasses?
> > > > > > > > Or is there a way to just register the Card object and have it still
> > > > > > > > able to send the subclasses?
>
> > > > > > > > --
> > > > > > > > You received this message because you are subscribed to the
> > > > > > > > "kryonet-users" group.
> > > > > > > >http://groups.google.com/group/kryonet-users
>
> > > > > > > --
> > > > > > > Sam Ritchie, Twitter Inc
> > > > > > > @sritchie09
>
> > > > > > > (Too brief? Here's why!http://emailcharter.org)
>
> > > > > > --
> > > > > > You received this message because you are subscribed to the "kryonet-users" group.
> > > > > >http://groups.google.com/group/kryonet-users
>
> > > > --
> > > > You received this message because you are subscribed to the "kryonet-users" group.
> > > >http://groups.google.com/group/kryonet-users
>
> > > --
> > > Sam Ritchie, Twitter Inc
> > > 703.662.1337 (tel:703.662.1337)
>
> > > > > > > (Too brief? Here's why!http://emailcharter.org)
>
> > > > > > --
> > > > > > You received this message because you are subscribed to the "kryonet-users" group.
> > > > > >http://groups.google.com/group/kryonet-users
>
> > > > --
> > > > You received this message because you are subscribed to the "kryonet-users" group.
> > > >http://groups.google.com/group/kryonet-users
>
> > > --
> > > Sam Ritchie, Twitter Inc
Jonah Stiennon
May 9, 2019, 3:02:46 PM5/9/19
to kryonet-users
I belive the original problem here can now be easily solved using `Kryo.addDefaultSerializer(...)`. Correct?
Reply all
Reply to author
Forward
0 new messages