Throwable and Suppressed Exceptions

[Robert DiFalco]

I think I'm having an issue with serializing throwable objects but I can't be sure. Should I have to do anything special for them besides settings references on to handle self referencing causes?

Here's my issue -- and this is in a debugger which is why I saw it. If I deserialize a Throwable with Kryo and then inspect the deserialized exception I get a NullPointerException on "suppressedExceptions" which is meant to be an UnmodifiableRandomAccessList of length 0. I'm not sure where the problem is or how to dig into it. 

Anyone experienced this?

Thanks!

[Robert DiFalco]

Here's what the exception looks like if I tried to print the stack trace:

com.myapp.service.shared.ServiceException: This should throw an exception.

at com.myapp.job.server.SimpleJobTest$TestServerImpl.throwException(SimpleJobTest.java:117)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:601)

at com.myapp.job.server.JobServerMethod.execute(JobServerMethod.java:30)

at com.myapp.job.server.JobExecution.evaluate(JobExecution.java:54)

at com.myapp.job.server.JobExecution.run(JobExecution.java:39)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)

at java.lang.Thread.run(Thread.java:722)

java.lang.NullPointerException

at java.util.Collections$UnmodifiableCollection.toArray(Collections.java:1059)

at java.lang.Throwable.getSuppressed(Throwable.java:1078)

at java.lang.Throwable.printStackTrace(Throwable.java:660)

at java.lang.Throwable.printStackTrace(Throwable.java:642)

at java.lang.Throwable.printStackTrace(Throwable.java:633)

at com.myapp.job.server.SimpleJobTest.jobThrowsException(SimpleJobTest.java:84)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:601)

at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)

at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)

at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)

at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)

at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)

at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:74)

[Nate]

Show your serializer? I doubt FieldSerializer can/should handle exceptions.

-Nate

--
You received this message because you are subscribed to the "kryo-users" group.
http://groups.google.com/group/kryo-users
---
You received this message because you are subscribed to the Google Groups "kryo-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kryo-users+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Robert DiFalco]

Ah, I'm sure that's my issue then. How do most people deal with exceptions. I suppose I thought there would be a default serializer for it. It wasn't clear to me that I should do special wiring for Kryo to handle it. This seems like it did the trick, or should I be using a different approach?

        kryo.addDefaultSerializer( Throwable.class, new JavaSerializer() );

Also, is there a default way to add the JavaSerializer for Serializable.class in such a way that it has the LOWEST priority? Ideally I would like it to be a last resort and to LOG that it is using it so I can clean them up as needed. I have way to many classes to handle to explicitly add a registration for each one since this is a pretty generic system whose legacy requirement was only that classes be Serializable.

Thanks!

You received this message because you are subscribed to a topic in the Google Groups "kryo-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kryo-users/9xOngIaZdO4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kryo-users+...@googlegroups.com.

[Nate]

On Sun, Jan 5, 2014 at 8:48 PM, Robert DiFalco <robert....@gmail.com> wrote:

Ah, I'm sure that's my issue then. How do most people deal with exceptions. I suppose I thought there would be a default serializer for it. It wasn't clear to me that I should do special wiring for Kryo to handle it.

FieldSerializer serializes private fields. If those are private API, it may not be intended for something external to much with, so sometimes it doesn't work very well. The alternative is writing your own, non-generic serializer that uses only public API.

 

This seems like it did the trick, or should I be using a different approach?

        kryo.addDefaultSerializer( Throwable.class, new JavaSerializer() );

That may be fine for you, though the Java serializer tends to output many bytes.

 

Also, is there a default way to add the JavaSerializer for Serializable.class in such a way that it has the LOWEST priority? Ideally I would like it to be a last resort and to LOG that it is using it so I can clean them up as needed. I have way to many classes to handle to explicitly add a registration for each one since this is a pretty generic system whose legacy requirement was only that classes be Serializable.

The priority is the order they are added, except the serializers added by default are lower than any added manually. I'm afraid there is no way to make a manually added serializer lower priority than the default (you could edit Kryo.java). Subclass JavaSerializer if you want a log message.

-Nate

[Tumi]

Shouldn't be addDefaultSerializer(Throwable.class, JavaSerializer.class) added in Kryo constructor?
I double checked that FieldSerializer generates NullPointerExceptions (and StackOverflow if not resolving references).

Looking at the Throwable.java source code it doesn't seem to be generate an efficent serializer without reflection magic.

I suggest to register in Kryo the default JavaSerializer for any incompatible cases as this that in the future can be detected.

[Robert DiFalco]

This is another way to fix it so it will work without resorting to Java Serialization.

        kryo.addDefaultSerializer(

            Collections.unmodifiableCollection( Collections.EMPTY_LIST ).getClass(),

            UnmodifiableCollectionsSerializer.class );

[Jesús Viñuales]

UnmodifiableCollectionsSerializer?

You received this message because you are subscribed to the Google Groups "kryo-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to kryo-users+...@googlegroups.com.

[Robert DiFalco]

It's from javakaffee's extra Kryo Serializers.

[Jesús Viñuales]

[Martin Grotzke]

Slightly related:

https://github.com/EsotericSoftware/kryo/issues/151 - "Move serializers
for std jdk classes from kryo-serializers to kryo"

It's on my list but I won't find time for it the next two weeks because
I'm on vacation next week.

Cheers,
Martin

> <mailto:kryo-users%2Bunsu...@googlegroups.com>.

> For more options, visit https://groups.google.com/groups/opt_out.
>
>

> --
> You received this message because you are subscribed to the "kryo-users"
> group.
> http://groups.google.com/group/kryo-users
> ---

> You received this message because you are subscribed to the Google
> Groups "kryo-users" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to kryo-users+...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

--
inoio gmbh - http://inoio.de
Schulterblatt 36, 20357 Hamburg
Amtsgericht Hamburg, HRB 123031
Geschäftsführer: Dennis Brakhane, Martin Grotzke, Ole Langbehn

[Nate]

On Tue, Jan 7, 2014 at 2:55 AM, Robert DiFalco <robert....@gmail.com> wrote:

It's from javakaffee's extra Kryo Serializers.

I'm open to including serializers that are JVM specific, but they shouldn't be registered by default. I haven't checked UnmodifiableCollectionsSerializer but I expect it uses reflection.

Serializing Throwable is like serializing Object, we have no idea what subclasses may have added. Each throwable implementation needs to be considered individually.

-Nate

[Robert DiFalco]

Nate, is anything that uses Reflection interfaces JVM specific? Or just those that access private methods?

[Nate]

On Tue, Jan 7, 2014 at 6:05 PM, Robert DiFalco <robert....@gmail.com> wrote:

Nate, is anything that uses Reflection interfaces JVM specific? Or just those that access private methods?

Accessing a private API with reflection means your code may not work if the implementation is different or changes. Reflection itself is fine.

-Nate

[Robert DiFalco]

Check. Just making sure since I am working on an experiment right now to fully support ObjectInput and ObjectOutput with a Kryo adapter. Unfortunately, that means I need to use reflection to get readObject, writeObject, realResolve, and writeReplace. While those are private they are VERY stable implementation details. 

If I'm successful, it means there is a very efficient approach to using serializable/externalizable objects with Kryo. Should be a better alternative to JavaSerializer. 

But if you've already tried to go down this path, I can already tell you I can see why you stopped! Sun made this a royal pain in the ass! It's like they didn't even know what they were doing when they wrote this stuff. All the special cases, proxy, non-proxy handling, etc. 

One path I'm going down that feels good is putting the readReplace and writeReplace handling in the ExternalizableSerializer and NOT in the KryoObjectOutput#writeObject and  KryoObjectInput#readObject adapters. This way other types with registrations need not have that introspection. 

But I may not derp on this for very long, just giving it a try.  So no pull request is imminent. I'm thinking you'll reject it anyway and request it go into a kryo add-on jar because of all the stupid hackery I have to do to make it work.

[Nate]

On Tue, Jan 7, 2014 at 6:37 PM, Robert DiFalco <robert....@gmail.com> wrote:

Check. Just making sure since I am working on an experiment right now to fully support ObjectInput and ObjectOutput with a Kryo adapter. Unfortunately, that means I need to use reflection to get readObject, writeObject, realResolve, and writeReplace. While those are private they are VERY stable implementation details. 

I don't think any of those methods are relevant. An ObjectInput/ObjectOutput for an ExternalizableSerializer would delegate to an Input/Output and a Kryo instance, nothing more.

-Nate

[Robert DiFalco]

Well, it would make things a lot simpler. I guess I wanted an efficient way for types like "Locale" to JUST WORK that use readResolve or writeReplace. I had to register to the JavaSerializer for Locale and it makes me unhappy to ever have to rely on this serializer. Seems if I could support the full java serialization API then any non-kryo object would work using the kryo stream protocol. Efficiency with the correct behavior. I wouldn't have to discover all these issues one by one like Throwable or Locale.

[Nate]

On Tue, Jan 7, 2014 at 7:07 PM, Robert DiFalco <robert....@gmail.com> wrote:

Well, it would make things a lot simpler. I guess I wanted an efficient way for types like "Locale" to JUST WORK that use readResolve or writeReplace. I had to register to the JavaSerializer for Locale and it makes me unhappy to ever have to rely on this serializer. Seems if I could support the full java serialization API then any non-kryo object would work using the kryo stream protocol. Efficiency with the correct behavior. I wouldn't have to discover all these issues one by one like Throwable or Locale.

Thar be dragons. You'll want to avoid the Java serialization quagmire. :)

-Nate

[Robert DiFalco]

Yeah, I think you are write. Yeesh, defaultWriteObject? Ugh.

Maybe a branched serializer! If I detect anything like readReplace or writeReplace I construct and cache a JavaSerializer and just fall back on that. :) Since the detection of those methods slows things down a little (but I will cache them for the type) I could make the serializer take a constructor argument to enable or disable that check. 

Might even make a nice addition to FieldSerializer? It seems like FieldSerializer only fails on Java objects that use stupid serialization tricks with read or write replace.