Migration to Kryo 3.x and Storm API 1.0.x

6 views
Skip to first unread message

Timur Abishev

unread,
Nov 7, 2016, 6:11:10 PM11/7/16
to Scalding Development
We are going to move to Kryo 3.x and 1.0.x Storm API at Twitter. To accomplish that we need to migrate: 1) chill-storm to 1.0.x, 2) tormenta to storm 1.0.x and chill 0.8.x, 3) scalding to chill 0.8.x and finally 4) summingbird to new chill, tormenta and scalding.

Any suggestions/objections about that?

--
Timur Abishev

P. Oscar Boykin

unread,
Nov 7, 2016, 9:21:21 PM11/7/16
to Timur Abishev, Scalding Development
This plan seems correct to me.

It is worth considering if we should look into shading Kryo in scalding. Having this many projects coupled on a version is not great.

--
You received this message because you are subscribed to the Google Groups "Scalding Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scalding-dev...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Alex Levenson

unread,
Nov 7, 2016, 9:27:21 PM11/7/16
to P. Oscar Boykin, Timur Abishev, Scalding Development
Yeah, that's a good question. In some ways I think SB and Storm sort of communicate through kryo though right? They share a kryo registrar, it's how users setup their serialization in both frameworks right?

To unsubscribe from this group and stop receiving emails from it, send an email to scalding-dev+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Scalding Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scalding-dev+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
Alex Levenson
@THISWILLWORK

P. Oscar Boykin

unread,
Nov 8, 2016, 11:18:08 AM11/8/16
to Alex Levenson, Timur Abishev, Scalding Development
Yeah, in summingbird there is an expectation it can configure Kryo. It is a hard problem.
On Mon, Nov 7, 2016 at 16:27 'Alex Levenson' via Scalding Development <scaldi...@googlegroups.com> wrote:
Yeah, that's a good question. In some ways I think SB and Storm sort of communicate through kryo though right? They share a kryo registrar, it's how users setup their serialization in both frameworks right?
On Mon, Nov 7, 2016 at 6:21 PM, P. Oscar Boykin <oscar....@gmail.com> wrote:
This plan seems correct to me.

It is worth considering if we should look into shading Kryo in scalding. Having this many projects coupled on a version is not great.
On Mon, Nov 7, 2016 at 1:11 PM 'Timur Abishev' via Scalding Development <scaldi...@googlegroups.com> wrote:
We are going to move to Kryo 3.x and 1.0.x Storm API at Twitter. To accomplish that we need to migrate: 1) chill-storm to 1.0.x, 2) tormenta to storm 1.0.x and chill 0.8.x, 3) scalding to chill 0.8.x and finally 4) summingbird to new chill, tormenta and scalding.

Any suggestions/objections about that?

--
Timur Abishev

--
You received this message because you are subscribed to the Google Groups "Scalding Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scalding-dev...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Scalding Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scalding-dev...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
Alex Levenson
@THISWILLWORK

--
You received this message because you are subscribed to the Google Groups "Scalding Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scalding-dev...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages