2.22 estimate?

[Oscar Boykin]

Some of the known issues in 2.21 are biting us now, and we can't float our production jobs on 2.22-SNAPSHOT.

Any ETA on 2.22?

Could we cut a 2.22 final with what we have now and fix any issues in 2.23?

--

Oscar Boykin :: @posco :: http://twitter.com/posco

[Martin Grotzke]

+1

--
You received this message because you are subscribed to the "kryo-users" group.
http://groups.google.com/group/kryo-users
---
You received this message because you are subscribed to the Google Groups "kryo-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kryo-users+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[mongonix]

In principle, I like the idea.

At the same time, I also see some reasons why we should be very careful when releasing 2.22.

Pros:

- 2.22 introduces new Unsafe-based  IO streams, which are much faster than usual IO streams  (this approach is already ported from Kryo into Avro and Hazelcast 3.0)

- 2.22 introduces a new Unsafe-based FieldSerializer and array serializers, which are much faster than the old Kryo serializers (this approach is already ported from Kryo into Avro and Hazelcast 3.0)

- 2.22 introduces support for serializing very fast and direcy into the off-heap memory

- A lot of reported issues were fixed

Cons:

- There are very many significant changes in the Kryo code-based since 2.21. A lot of re-factoring was done. I lot of new code was added. The amount of changes is much bigger than it was usually the case

- Unsafe-based improvements seems to be pretty stable, but are not widely tested in the wild by Kryo users yet.

- Documentation for the new Unsafe-based Kryo mechanisms does not exisit yet (only JavaDocs)

- issue 115 (https://code.google.com/p/kryo/issues/detail?id=115) seems to be a serious bug, which I'd like to be fixed before releasing

- shaded and OSGI packaging issues are not solves yet, IIRC

Having said all that I also realize that many people won't test any new features until they are provided by an official release of Kryo, because only a few are brave enough to experiment with latest snapshots. And when going into production, almost nobody wants to use snapshot versions. Therefore, the only way to spread the new features and get them tested is to provide a release containing them.

What do you think about pros and cons described above? How important are they in your opinion? 

-Leo

[Martin Grotzke]

Basically I think we could release a 2.22-RC1.

On 09/06/2013 08:33 AM, mongonix wrote:
> Cons:
> - There are very many significant changes in the Kryo code-based since
> 2.21. A lot of re-factoring was done. I lot of new code was added. The
> amount of changes is much bigger than it was usually the case
> - Unsafe-based improvements seems to be pretty stable, but are not
> widely tested in the wild by Kryo users yet.

To get more feedback / confidence re stability a non-SNAPSHOT version helps.

> - Documentation for the new Unsafe-based Kryo mechanisms does not exisit
> yet (only JavaDocs)
> - issue 115 (https://code.google.com/p/kryo/issues/detail?id=115) seems
> to be a serious bug, which I'd like to be fixed before releasing

Sounds reasonable, maybe the RC announement could mention this as a
known-issue?

> - shaded and OSGI packaging issues are not solves yet, IIRC

For the RC I can resolve the issue with the shaded jar, I'd make the
shaded jar the main jar (so that there's only the "normal" jar artifact
and this contains all relocated deps).

For the final release I'd checkout the OSGI packaging.

> Having said all that I also realize that many people won't test any new
> features until they are provided by an official release of Kryo, because
> only a few are brave enough to experiment with latest snapshots. And
> when going into production, almost nobody wants to use snapshot
> versions. Therefore, the only way to spread the new features and get
> them tested is to provide a release containing them.

IMHO it's good to cut a release candidate, mention the major changes and
list all issues that should be resolved until the release. E.g. this
could be done on the project main page.
Then bugs can be fixed as reported. Thanx to your current engagement I'm
confident that this will happen - assuming that you can take the time
once the RC/release is out :-)

If people are running into serious problems they should be able to go
back to 2.21. Or did s.th. regarding serialization change so that
serialized data is not compatible?

Cheers,
Martin

>
> What do you think about pros and cons described above? How important are
> they in your opinion?
>
> -Leo
>
> On Friday, September 6, 2013 12:53:50 AM UTC+2, Oscar Boykin wrote:
>
> Some of the known issues in 2.21 are biting us now, and we can't
> float our production jobs on 2.22-SNAPSHOT.
>
> Any ETA on 2.22?
>
> Could we cut a 2.22 final with what we have now and fix any issues
> in 2.23?
> --
> Oscar Boykin :: @posco :: http://twitter.com/posco
>

> --
> You received this message because you are subscribed to the "kryo-users"
> group.
> http://groups.google.com/group/kryo-users
> ---
> You received this message because you are subscribed to the Google
> Groups "kryo-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to kryo-users+...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

--
inoio gmbh - http://inoio.de
Schulterblatt 36, 20357 Hamburg
Amtsgericht Hamburg, HRB 123031
Geschäftsführer: Dennis Brakhane, Martin Grotzke, Ole Langbehn

[mongonix]

On Friday, September 6, 2013 10:35:44 AM UTC+2, Martin Grotzke wrote:

Basically I think we could release a 2.22-RC1.

On 09/06/2013 08:33 AM, mongonix wrote:
> Cons:
> - There are very many significant changes in the Kryo code-based since
> 2.21. A lot of re-factoring was done. I lot of new code was added. The
> amount of changes is much bigger than it was usually the case
> - Unsafe-based improvements seems to be pretty stable, but are not
> widely tested in the wild by Kryo users yet.

To get more feedback / confidence re stability a non-SNAPSHOT version helps.

> - Documentation for the new Unsafe-based Kryo mechanisms does not exisit
> yet (only JavaDocs)
> - issue 115 (https://code.google.com/p/kryo/issues/detail?id=115) seems
> to be a serious bug, which I'd like to be fixed before releasing

Sounds reasonable, maybe the RC announement could mention this as a
known-issue?

Yes. This could be an idea.

 

> - shaded and OSGI packaging issues are not solves yet, IIRC

For the RC I can resolve the issue with the shaded jar, I'd make the
shaded jar the main jar (so that there's only the "normal" jar artifact
and this contains all relocated deps).

If you look into it, can you also check why shaded jars also include the original jars of dependencies besides the relocated classes? I don't quite understand why this should be the case.

BTW, in theory, if only Unsafe-based FieldSerializer is to be used on certain platforms, we could probably completely remove ASM-dependency from a Kryo packaging for such platforms.

 

For the final release I'd checkout the OSGI packaging.

Nice.

 

> Having said all that I also realize that many people won't test any new
> features until they are provided by an official release of Kryo, because
> only a few are brave enough to experiment with latest snapshots. And
> when going into production, almost nobody wants to use snapshot
> versions. Therefore, the only way to spread the new features and get
> them tested is to provide a release containing them.

IMHO it's good to cut a release candidate, mention the major changes and
list all issues that should be resolved until the release. E.g. this
could be done on the project main page.

Good idea. 

@Nate: Can you give me and Martin the permissions to edit project pages?

 

Then bugs can be fixed as reported. Thanx to your current engagement I'm
confident that this will happen - assuming that you can take the time
once the RC/release is out :-)

At the moment it looks like I'll have the time for it. 

 

If people are running into serious problems they should be able to go
back to 2.21. Or did s.th. regarding serialization change so that
serialized data is not compatible?

No, I think that non-Unsafe-based implementations are backwards compatible. And the new Unsafe-based implementations are intentionally not compatible, which should be explicitly stated in JavaDocs and on the project home page.

@Nate: What do you think? Should we release a 2.22-RC1 and soon after a final release of 2.22?

Cheers,

  Leo

[Martin Grotzke]

On 09/06/2013 11:13 AM, mongonix wrote:
> On Friday, September 6, 2013 10:35:44 AM UTC+2, Martin Grotzke wrote:
> > - shaded and OSGI packaging issues are not solves yet, IIRC
>
> For the RC I can resolve the issue with the shaded jar, I'd make the
> shaded jar the main jar (so that there's only the "normal" jar artifact
> and this contains all relocated deps).
>
>
> If you look into it, can you also check why shaded jars also include the
> original jars of dependencies besides the relocated classes? I don't
> quite understand why this should be the case.

Yes, I'll check this.

> BTW, in theory, if only Unsafe-based FieldSerializer is to be used on
> certain platforms, we could probably completely remove ASM-dependency
> from a Kryo packaging for such platforms.

I don't think that it's worth providing platform-specific packages: the
only advantage I see is a smaller jar (< 10k diff), while the choice for
users is more difficult and you the build gets more complex.
Do you see other advantages?

> Then bugs can be fixed as reported. Thanx to your current engagement
> I'm
> confident that this will happen - assuming that you can take the time
> once the RC/release is out :-)
>
> At the moment it looks like I'll have the time for it.

Great!

> If people are running into serious problems they should be able to go

> back to 2.21. Or did s.th <http://s.th>. regarding serialization

> change so that
> serialized data is not compatible?
>
>
> No, I think that non-Unsafe-based implementations are backwards
> compatible. And the new Unsafe-based implementations are intentionally
> not compatible, which should be explicitly stated in JavaDocs and on the
> project home page.

+1

Cheers,
Martin

> > an email to kryo-users+...@googlegroups.com <javascript:>.

> > For more options, visit https://groups.google.com/groups/opt_out

> <https://groups.google.com/groups/opt_out>.

[mongonix]

On Friday, September 6, 2013 1:50:43 PM UTC+2, Martin Grotzke wrote:

> BTW, in theory, if only Unsafe-based FieldSerializer is to be used on
> certain platforms, we could probably completely remove ASM-dependency
> from a Kryo packaging for such platforms.

I don't think that it's worth providing platform-specific packages: the
only advantage I see is a smaller jar (< 10k diff), while the choice for
users is more difficult and you the build gets more complex.
Do you see other advantages?

Not really ;-) It was meant more for the sake of making those people happy, who use a different version of ASM and are afraid of possible conflicts. But with shaded ASM it is not a problem anyway.

 

Cheers,

  Leo

[mongonix]

Hi Oscar,

On Friday, September 6, 2013 12:53:50 AM UTC+2, Oscar Boykin wrote:

Some of the known issues in 2.21 are biting us now, and we can't float our production jobs on 2.22-SNAPSHOT.

BTW, does it imply that you checked and those known issues in 2.21 are fixed in 2.22-SNAPSHOT already?

-Leo

[Joachim Durchholz]

Am 06.09.2013 14:07, schrieb mongonix:
> Not really ;-) It was meant more for the sake of making those people happy,
> who use a different version of ASM and are afraid of possible conflicts.
> But with shaded ASM it is not a problem anyway.

There's still the problem that ASM-generated code is opaque for most
people while debugging.
It's usually just an annoyance, but it can get really nasty if generated
code calls back into user code, or if inheritance comes into play. I
don't know whether either of these things can happen with Kryo; if not,
it's a mere annoyance.

For my specific use cases, the performance gain of ASM-generated code is
noticeably less valuable than the code transparency I'm losing here.
Just providing this as a data point, I won't whine and complain if you
don't remove ASM :-)

Regards,
Jo

[Oscar Boykin]

On Fri, Sep 6, 2013 at 7:23 AM, mongonix <romi...@gmail.com> wrote:

BTW, does it imply that you checked and those known issues in 2.21 are fixed in 2.22-SNAPSHOT already?

Looks like using 2.22 does fix at least a few of the issues we saw and were blockers. I haven't tested one other case (I just put in code to fall back to Java serialization there since speed is not the concern in that code, robustness of serialization is the concern).

 

-Leo

 

Any ETA on 2.22?

Could we cut a 2.22 final with what we have now and fix any issues in 2.23?

--

Oscar Boykin :: @posco :: http://twitter.com/posco

--

You received this message because you are subscribed to the "kryo-users" group.
http://groups.google.com/group/kryo-users

---
You received this message because you are subscribed to the Google Groups "kryo-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to kryo-users+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Brock Noland]

Hi,

 

Any ETA on 2.22?

Could we cut a 2.22 final with what we have now and fix any issues in 2.23?

+1, over in the Apache Hive project we are working using Kryo (https://issues.apache.org/jira/browse/HIVE-1511) but won't be able to ship the change until a v2.22 is released.

Brock 

[Sam Ritchie]

Wow, interesting. I wonder if we could get you guys to consider using Kryo via Chill:

https://github.com/twitter/chill

Spark, Scalding, Summingbird and Cascalog (soon) are all dependent on Chill, so the community is great. The win you'd get by pulling in Kryo via Chill is an ability to easily configure Serializers of all kinds via standard JobConf entries. This would help clusters that share Hive with other DSLs to stay consistent.

Oscar and I would be happy to contribute to that ticket if you think the community would be open.

Brock Noland

September 7, 2013 3:29 PM

Hi,

 

+1, over in the Apache Hive project we are working using Kryo (https://issues.apache.org/jira/browse/HIVE-1511) but won't be able to ship the change until a v2.22 is released.

Brock 

--
You received this message because you are subscribed to the "kryo-users" group.
http://groups.google.com/group/kryo-users
---
You received this message because you are subscribed to the Google Groups "kryo-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kryo-users+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Oscar Boykin

September 5, 2013 3:53 PM

Some of the known issues in 2.21 are biting us now, and we can't float our production jobs on 2.22-SNAPSHOT.

Any ETA on 2.22?

Could we cut a 2.22 final with what we have now and fix any issues in 2.23?

--

Oscar Boykin :: @posco :: http://twitter.com/posco

--
You received this message because you are subscribed to the "kryo-users" group.
http://groups.google.com/group/kryo-users
---
You received this message because you are subscribed to the Google Groups "kryo-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kryo-users+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Sam Ritchie, Twitter Inc
703.662.1337
@sritchie

[Brock Noland]

Hi Sam,

Chill looks interesting!  I think we should look at introducing Chill in another JIRA.  We've almost got 1511 wrapped up, it gives us good benefit as-is, and only use it for query plan serialization as opposed to data serialization.  I think Chill could be useful there, but it looks like it's real benefits is in data serialization.  Would you like to open a new jira describing how chill could be beneficial?

Thank you for reaching out!

Brock

[mongonix]

Hi Brock,

On Sunday, September 8, 2013 5:41:57 PM UTC+2, Brock Noland wrote:

Hi Sam,

Chill looks interesting!  I think we should look at introducing Chill in another JIRA.  We've almost got 1511 wrapped up, it gives us good benefit as-is, and only use it for query plan serialization as opposed to data serialization.  

Now that all tests almost pass using patches from 1511, one can look into performance issues, because current patches are not optimized for speed at all. I've commented on the issue already about possible optimizations (e.g. Kryo instances reuse, etc). Let me know if you need any help with this.

-Leo

[Brock Noland]

Hi,

On Sunday, September 8, 2013 11:44:34 AM UTC-5, mongonix wrote:

Hi Brock,

On Sunday, September 8, 2013 5:41:57 PM UTC+2, Brock Noland wrote:

Hi Sam,

Chill looks interesting!  I think we should look at introducing Chill in another JIRA.  We've almost got 1511 wrapped up, it gives us good benefit as-is, and only use it for query plan serialization as opposed to data serialization.  

Now that all tests almost pass using patches from 1511, one can look into performance issues, because current patches are not optimized for speed at all. I've commented on the issue already about possible optimizations (e.g. Kryo instances reuse, etc). Let me know if you need any help with this

Thank you so much for all the help you have given us!!  My first priority at the moment is stabilizing the build on Java 7 (which requires Kryo 2.22), Hadoop 2, and HBase 0.96 (the unholy trinity). After we get that done I think we'll spend some time looking at performance.

Thanks again!

Brock

[Oscar Boykin]

People,

Losing my mind hacking around known bugs in 2.21.

Is 2.22 coming soon or should we fork and drive our own boat?

--

You received this message because you are subscribed to the "kryo-users" group.
http://groups.google.com/group/kryo-users
---
You received this message because you are subscribed to the Google Groups "kryo-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to kryo-users+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[mongonix]

Hi,

Nate has not reacted to this thread at all yet. I have even created a dedicated isssue https://code.google.com/p/kryo/issues/detail?id=133 for releasing 2.22 and CCed Nate explicitly.

But we don't have any reply so far. For whatever reason Nate does not react. I'm not even sure he reads our messages...

Without him we cannot release 2.22 for two reasons:

1) We simply do not have enough permissions on Kryo's GoogleCode project to do this, because Nate has not given us all required permissions. I asked for more permissions, but haven't got any reply yet.

2) Nate is the maintainer and original author of Kryo and it would be nice if new releases are at least blessed (if not done) by him.

Forking Kryo would not be so nice mid-term or long-term and should be used only as a last resort, IMHO. But I totally understand your sentiment about releasing 2.22. The current situation is not exactly satisfactory ;-(

Let's wait a bit more (let's say 1 week) and hope that Nate finally reads his mails. If not, we can consider forking more seriously. Does it sound like a plan?

-Leo

[Martin Grotzke]

On 09/26/2013 10:28 AM, mongonix wrote:
> Without him we cannot release 2.22 for two reasons:
> 1) We simply do not have enough permissions on Kryo's GoogleCode project
> to do this, because Nate has not given us all required permissions. I
> asked for more permissions, but haven't got any reply yet.

I agree we should wait a bit more. Independently, which permissions are
you missing? We can push/commit, and I'm regularly making the release to
sonatype.

Cheers,
Martin

[mongonix]

Yes, I can commit push/commit. But for example, I cannot edit the project page to update the docs. I think I also tried to branch/tag and this also does not work for me. And it is a bit difficult to make a release without being able to perform these actions, or ;-)

But may be your have enough permissions, Martin?

 

Cheers,

  Roman

[Martin Grotzke]

On 09/26/2013 01:55 PM, mongonix wrote:
> Yes, I can commit push/commit. But for example, I cannot edit the
> project page to update the docs. I think I also tried to branch/tag and
> this also does not work for me. And it is a bit difficult to make a
> release without being able to perform these actions, or ;-)
>
> But may be your have enough permissions, Martin?

I can also update the project page.

Cheers,
Martin

[Nate]

By all means, don't hesitate to do a new release. If it breaks, people will revert to an older version. In the mean time, fix things quickly and do another release. This is much better than sitting on good code that people could be using. :)

I've uploaded a 2.22 JAR to Google Code and made an SVN tag. Martin, can you release a 2.22 Maven build?

I've made Leo and Martin owners. This allows you to change the homepage text (project description) on the Administer tab. Please be careful, there is no history for the main page text.

SVN access is full access, you should be able to do everything.

I have not had time to be active on the project for some time, but I am still interested in Kryo. Forking it would not be good for the library IMO.

-Nate

--

[mongonix]

Hi Nate,

You are back! Nice to hear from you!!! :-)

On Monday, September 30, 2013 1:26:48 AM UTC+2, Nate wrote:

By all means, don't hesitate to do a new release. If it breaks, people will revert to an older version. In the mean time, fix things quickly and do another release. This is much better than sitting on good code that people could be using. :)

I've uploaded a 2.22 JAR to Google Code and made an SVN tag. Martin, can you release a 2.22 Maven build?

Very nice! 

Regarding the release of 2.22:

- I think we are more or less ready to release a binary JAR, as I don't expect any planned changes there. This would already allow other projects to use the official 2.22 release.

- Martin, would you have time to check the OSGI packaging and shading issues we discussed before?

- When it comes to updating the docs, I'd really like to update it with infos about Unsafe-based IO streams, Unsafe-based FieldSerializers, etc. But unfortunately, I'll be unavailable for 2 or 4 weeks starting next Monday. I'll try to write something this week, if I find a free slot. But in the worst case, I'll have to update the docs on the project page once I'm available again. Is it OK? I don't think anyone started suing Unsafe-based and other new features of 2.22 already. Therefore delaying a bit the publishing of docs should not hurt anyone that much.

 

I've made Leo and Martin owners. This allows you to change the homepage text (project description) on the Administer tab. Please be careful, there is no history for the main page text. 

SVN access is full access, you should be able to do everything.

Thanks!

 

I have not had time to be active on the project for some time, but I am still interested in Kryo. Forking it would not be good for the library IMO.

Don't worry. Martin and I have the same opinion.

-Leo

[Martin Grotzke]

I just pushed 2.22 to sonatype, it will be in maven central in a few hours.

On 09/30/2013 08:48 AM, mongonix wrote:
> Regarding the release of 2.22:

> - Martin, would you have time to check the OSGI packaging and shading
> issues we discussed before?

I resolved the shading issue as discussed: the shaded jar is now the
main artifact, with all dependencies included (relocated). That the jar
additionally includes other jars (e.g. objenesis) is caused by the
configured osgi bundling (mvn-bundle-plugin), so it's expected I'd say.

Which was the OSGI packaging issue again you're refering to? I'd like to
take more time for this or check with s.o. more familiar with osgi.

> - When it comes to updating the docs, I'd really like to update it with
> infos about Unsafe-based IO streams, Unsafe-based FieldSerializers, etc.
> But unfortunately, I'll be unavailable for 2 or 4 weeks starting next
> Monday. I'll try to write something this week, if I find a free slot.
> But in the worst case, I'll have to update the docs on the project page
> once I'm available again. Is it OK? I don't think anyone started suing
> Unsafe-based and other new features of 2.22 already. Therefore delaying
> a bit the publishing of docs should not hurt anyone that much.

Ok. It would be nice to announce the release of 2.22 on the home page
("New!" section), with just some words. Are you going to do this?

Cheers,
Martin

[mongonix]

On Tuesday, October 1, 2013 12:03:06 AM UTC+2, Martin Grotzke wrote:

I just pushed 2.22 to sonatype, it will be in maven central in a few hours.

Thanks!

 

On 09/30/2013 08:48 AM, mongonix wrote:
> Regarding the release of 2.22:
> - Martin, would you have time to check the OSGI packaging and shading
> issues we discussed before?

I resolved the shading issue as discussed: the shaded jar is now the
main artifact, with all dependencies included (relocated). That the jar
additionally includes other jars (e.g. objenesis) is caused by the
configured osgi bundling (mvn-bundle-plugin), so it's expected I'd say.

Which was the OSGI packaging issue again you're refering to? I'd like to
take more time for this or check with s.o. more familiar with osgi.

I meant the issue 117 (https://code.google.com/p/kryo/issues/detail?id=117). If it is solved by what you just described, then we can close it.

 

> - When it comes to updating the docs, I'd really like to update it with
> infos about Unsafe-based IO streams, Unsafe-based FieldSerializers, etc.
> But unfortunately, I'll be unavailable for 2 or 4 weeks starting next
> Monday. I'll try to write something this week, if I find a free slot.
> But in the worst case, I'll have to update the docs on the project page
> once I'm available again. Is it OK? I don't think anyone started suing
> Unsafe-based and other new features of 2.22 already. Therefore delaying
> a bit the publishing of docs should not hurt anyone that much.

Ok. It would be nice to announce the release of 2.22 on the home page
("New!" section), with just some words. Are you going to do this?

I'll edit the home page today. I'm going to write a very short message about a new release and mention that it fixed quite some reported issues and introduced some of the features.

I'm also going to add on the top of the page a link to your snapshots repositories, because it is a rather popular question.

Cheers,

  Leo

[mongonix]

I updated the home page. Added a few words in the New section and added a very short section about Unsafe IO. Links to the snapshot repository were also added.

Please have a look and let me know if it is OK. Or may be something was forgotten or should be improved?

-Leo

[Martin Grotzke]

On 10/01/2013 09:16 AM, mongonix wrote:
> I meant the issue 117
> (https://code.google.com/p/kryo/issues/detail?id=117). If it is
> solved by what you just described, then we can close it.

Ok, there are some different opinions as it seems, I'll check with those
guys if they see possible improvements.

> I updated the home page. Added a few words in the New section and
> added a very short section about Unsafe IO. Links to the snapshot
> repository were also added.
>
> Please have a look and let me know if it is OK. Or may be something
> was forgotten or should be improved?

Great, thanx! I also added a note regarding the shaded jar.

Cheers,
Martin

[Ashutosh Chauhan]

So, I downloaded https://kryo.googlecode.com/files/kryo-2.22.zip and than used jars/production/onejar/kryo-2.22-all.jar With this jar in classpath I am hitting into following problem:

[junit] Exception in thread "main" java.lang.IncompatibleClassChangeError: Found interface org.objectweb.asm.MethodVisitor, but class was expected

[junit] at com.esotericsoftware.reflectasm.ConstructorAccess.insertConstructor(ConstructorAccess.java:89)

[junit] at com.esotericsoftware.reflectasm.ConstructorAccess.get(ConstructorAccess.java:70)

[junit] at com.esotericsoftware.kryo.Kryo.newInstantiator(Kryo.java:1058)

[junit] at com.esotericsoftware.kryo.Kryo.newInstance(Kryo.java:1109)

[junit] at com.esotericsoftware.kryo.serializers.FieldSerializer.create(FieldSerializer.java:526)

[junit] at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:502)

[junit] at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:672)

[junit] at org.apache.hadoop.hive.ql.exec.Utilities.deserializeObjectByKryo(Utilities.java:777)

Seems to me a packaging issue within Kryo. Is there any reason to believe that this would be better if I grab same Kryo jar from maven central. Is there any workaround for this problem?

Thanks,

Ashutosh

[Ashutosh Chauhan]

So, instead of kryo-2.22-all.jar I used kryo-2.22.jar and problem went away. I suspect this is because I somehow have old asm jar in my classpath. This seems to indicate shaded jar is not working as intentioned. If I find something more, I will update here.

[Martin Grotzke]

Hi Ashutosh,

the kryo-2.22-all.jar is not produced by maven / maven-shade-plugin. If
you want to get the shaded jar you can download it from maven central
(the default kryo-2.22.jar).

@Nate: can you shed some light on how the kryo-2.22-all.jar is produced?

Cheers,
Martin

> <https://code.google.com/p/kryo/issues/detail?id=117>). If it is

> > solved by what you just described, then we can close it.
>
> Ok, there are some different opinions as it seems, I'll check
> with those
> guys if they see possible improvements.
>
> > I updated the home page. Added a few words in the New section and
> > added a very short section about Unsafe IO. Links to the snapshot
> > repository were also added.
> >
> > Please have a look and let me know if it is OK. Or may be
> something
> > was forgotten or should be improved?
>
> Great, thanx! I also added a note regarding the shaded jar.
>
> Cheers,
> Martin
>
>

[Nate]

On Thu, Oct 3, 2013 at 6:35 PM, Martin Grotzke <martin....@googlemail.com> wrote:

Hi Ashutosh,

the kryo-2.22-all.jar is not produced by maven / maven-shade-plugin. If
you want to get the shaded jar you can download it from maven central
(the default kryo-2.22.jar).

@Nate: can you shed some light on how the kryo-2.22-all.jar is produced?

I run Scar on the project, copy the JARs from target/dist to debug JARs folder, replace the minlog JAR with the "none" version, run Scar, copy the JARs from target/dist to production JARs folder, export Javadocs from Eclipse making sure to uncheck the "test" source folder, and export the project source from the SVN using TortoiseSVN. Not fully automated but not much work and not done terribly often.

-Nate

[Martin Grotzke]

On 10/03/2013 06:42 PM, Nate wrote:
> I run Scar on the project, copy the JARs from target/dist to debug JARs
> folder, replace the minlog JAR with the "none" version, run Scar, copy
> the JARs from target/dist to production JARs folder, export Javadocs
> from Eclipse making sure to uncheck the "test" source folder, and export
> the project source from the SVN using TortoiseSVN. Not fully automated
> but not much work and not done terribly often.

From the http://code.google.com/p/scar/ homepage:
oneJar: Unzips multiple JARs and repackages them into a single JAR.

So the difference to the shaded jar we produce is that classes from e.g.
asm don't get relocated/renamed but are just packed into the single jar.

Cheers,
Martin