Groups
Sign in
Groups
Scalding Development
Conversations
About
Send feedback
Help
Scalding Development
Contact owners and managers
1–30 of 137
Mark all as read
Report group
0 selected
Christian Pernillo
2/21/23
Split one TypedPipe into multiple branches
Dear Scalding community, I'm struggling with a Scalding job that we need to optimize, the job
unread,
Split one TypedPipe into multiple branches
Dear Scalding community, I'm struggling with a Scalding job that we need to optimize, the job
2/21/23
Jing Lu
,
Alex Levenson
2
9/3/19
Read a gziped HIVE table in scalding
Hello, Not that I know of. If you can find a cascading Tap/Scheme that understands that file format,
unread,
Read a gziped HIVE table in scalding
Hello, Not that I know of. If you can find a cascading Tap/Scheme that understands that file format,
9/3/19
Jing Lu
,
Oscar Boykin
5
8/20/19
Is that possible to do exact "LATERAL VIEW EXPLODE" in scalding?
Yes, it does. Thanks for the explanation! Best On Tue, Aug 20, 2019 at 4:52 PM Oscar Boykin <oscar
unread,
Is that possible to do exact "LATERAL VIEW EXPLODE" in scalding?
Yes, it does. Thanks for the explanation! Best On Tue, Aug 20, 2019 at 4:52 PM Oscar Boykin <oscar
8/20/19
Saket Kumar
, …
Alex Levenson
8
5/23/19
Bucket join in Scalding
sorry I meant leftJoin not joinLeft On Thu, May 23, 2019 at 3:43 PM Alex Levenson <alexlevenson@
unread,
Bucket join in Scalding
sorry I meant leftJoin not joinLeft On Thu, May 23, 2019 at 3:43 PM Alex Levenson <alexlevenson@
5/23/19
ybro...@ebay.com
, …
Alex Levenson
4
4/25/19
Does scalding-parquet library support reading in snappy compressed Parquet files?
Parquet handles data encoding / compression a little bit different from most formats (it doesn't
unread,
Does scalding-parquet library support reading in snappy compressed Parquet files?
Parquet handles data encoding / compression a little bit different from most formats (it doesn't
4/25/19
Russell Carden
,
Oscar Boykin
3
1/15/19
Keys sorted on Reducers
Thank you. I see it on the wikipedia page for map reduce. On Monday, January 14, 2019 at 5:12:37 PM
unread,
Keys sorted on Reducers
Thank you. I see it on the wikipedia page for map reduce. On Monday, January 14, 2019 at 5:12:37 PM
1/15/19
Tianshan Cui
,
P. Oscar Boykin
3
8/28/18
Is there any good way to control the number of mappers for the sub-tasks in one scalding job?
Thanks for your quick response. That totally make sense. I guess the workaround in my case would be
unread,
Is there any good way to control the number of mappers for the sub-tasks in one scalding job?
Thanks for your quick response. That totally make sense. I guess the workaround in my case would be
8/28/18
Russell Carden
, …
Oscar Boykin
4
6/26/18
Reducer Estimators and GroupAll
yeah, that's pretty old.... 0.17.4 is the latest. Definitely worth upgrading. On Tue, Jun 26,
unread,
Reducer Estimators and GroupAll
yeah, that's pretty old.... 0.17.4 is the latest. Definitely worth upgrading. On Tue, Jun 26,
6/26/18
Jing Lu
, …
Oscar Boykin
11
6/23/18
How to profile code in scalding?
You can get an idea by how many records per second you are processing, how much total data you are
unread,
How to profile code in scalding?
You can get an idea by how many records per second you are processing, how much total data you are
6/23/18
王天宇
1/16/18
Is scalding a memory based framework like Spark or a wrapper for MR?
please.
unread,
Is scalding a memory based framework like Spark or a wrapper for MR?
please.
1/16/18
Kostya Salomatin
12/15/17
Scoring a matrix of elements
Hi scalding experts, I need an advise on a workflow to optimize my job efficiency. I need to score a
unread,
Scoring a matrix of elements
Hi scalding experts, I need an advise on a workflow to optimize my job efficiency. I need to score a
12/15/17
Cyrille Chépélov
,
charani...@gmail.com
3
12/4/17
Re: calculate the Distinct count for every field in the List at once
Would you mind explaining, why you opted flatMap instead on map for "DB.flatMap". thank you
unread,
Re: calculate the Distinct count for every field in the List at once
Would you mind explaining, why you opted flatMap instead on map for "DB.flatMap". thank you
12/4/17
mstr...@gmail.com
, …
Cyrille Chépélov
6
8/23/17
group by and apply the same reduce method on all (non-group by) fields
I was able to implement the logic I was looking for using the typed api, however, I believe Field-
unread,
group by and apply the same reduce method on all (non-group by) fields
I was able to implement the logic I was looking for using the typed api, however, I believe Field-
8/23/17
Russell Carden
,
Oscar Boykin
3
8/18/17
scanLeft and Arity
Since you said it was possible, I kept reading over the scalding's scanleft. I didn't find
unread,
scanLeft and Arity
Since you said it was possible, I kept reading over the scalding's scanleft. I didn't find
8/18/17
mstr...@gmail.com
, …
Koert Kuipers
12
7/1/17
getting/manupulating all fields in a pipe in scalding
That's exactly what I was looking for. Many thanks Koert, Oscar, and Alex. On Saturday, July 1,
unread,
getting/manupulating all fields in a pipe in scalding
That's exactly what I was looking for. Many thanks Koert, Oscar, and Alex. On Saturday, July 1,
7/1/17
Chris K Wensel
2/9/17
Cascading Community Updates
Hey all Sorry for the cross post, but I felt all three communities should be brought up to speed on
unread,
Cascading Community Updates
Hey all Sorry for the cross post, but I felt all three communities should be brought up to speed on
2/9/17
Nikhil J Joshi
, …
Alex Levenson
11
1/10/17
Small files not combined in mapper
If you look at how HfsConfPropertySetter is implemented, you just need to use a Tap that overrides
unread,
Small files not combined in mapper
If you look at how HfsConfPropertySetter is implemented, you just need to use a Tap that overrides
1/10/17
Nikhil J Joshi
, …
Kostya Salomatin
4
11/24/16
Adding distinct while joining to datasets raises exception
Thanks Piyush. That solved my problem. On Thursday, November 3, 2016 at 9:51:11 AM UTC-7, Piyush
unread,
Adding distinct while joining to datasets raises exception
Thanks Piyush. That solved my problem. On Thursday, November 3, 2016 at 9:51:11 AM UTC-7, Piyush
11/24/16
Nikhil J Joshi
11/24/16
TypedPipe to matrix
Hi, I have a library (XGBoost) that is optimized for matrix manipulation and hence I need to port my
unread,
TypedPipe to matrix
Hi, I have a library (XGBoost) that is optimized for matrix manipulation and hence I need to port my
11/24/16
Timur Abishev
, …
P. Oscar Boykin
4
11/8/16
Migration to Kryo 3.x and Storm API 1.0.x
Yeah, in summingbird there is an expectation it can configure Kryo. It is a hard problem. On Mon, Nov
unread,
Migration to Kryo 3.x and Storm API 1.0.x
Yeah, in summingbird there is an expectation it can configure Kryo. It is a hard problem. On Mon, Nov
11/8/16
Cyrille Chépélov
, …
Piyush Narang
10
11/4/16
[cascading3 branch] descriptions & parallelism
This is pretty cool! Shall take a look at the WIP PR as well. On Fri, Nov 4, 2016 at 12:57 PM,
unread,
[cascading3 branch] descriptions & parallelism
This is pretty cool! Shall take a look at the WIP PR as well. On Fri, Nov 4, 2016 at 12:57 PM,
11/4/16
og...@spotify.com
, …
Alex Levenson
8
10/26/16
Problems building Scalding and Running REPL locally
Invoking ./sbt (instead of `which sbt`) uses the version of sbt / scala / etc that scalding is setup
unread,
Problems building Scalding and Running REPL locally
Invoking ./sbt (instead of `which sbt`) uses the version of sbt / scala / etc that scalding is setup
10/26/16
Kostya Salomatin
,
Oscar Boykin
3
10/12/16
Does toTypedPipe call break some scalding optimization?
Thanks, that makes sense. On Wed, Oct 12, 2016 at 12:51 PM, 'Oscar Boykin' via Scalding
unread,
Does toTypedPipe call break some scalding optimization?
Thanks, that makes sense. On Wed, Oct 12, 2016 at 12:51 PM, 'Oscar Boykin' via Scalding
10/12/16
Gevorg Hari
, …
P. Oscar Boykin
4
9/28/16
Scalding on Stackoverflow
I really think that a better presence on Stackoverflow will simplify adoption and increase developers
unread,
Scalding on Stackoverflow
I really think that a better presence on Stackoverflow will simplify adoption and increase developers
9/28/16
Kostya Salomatin
,
Oscar Boykin
2
9/22/16
Serialization of internal job vals
It uses this code: https://github.com/twitter/chill/blob/develop/chill-scala/src/main/scala/com/
unread,
Serialization of internal job vals
It uses this code: https://github.com/twitter/chill/blob/develop/chill-scala/src/main/scala/com/
9/22/16
Koert Kuipers
,
P. Oscar Boykin
3
9/18/16
low priority implicits for TupleConverter and TupleSetter etc.
yeah agreed, it just caught me by surprise somewhat On Sun, Sep 18, 2016 at 5:48 PM, P. Oscar Boykin
unread,
low priority implicits for TupleConverter and TupleSetter etc.
yeah agreed, it just caught me by surprise somewhat On Sun, Sep 18, 2016 at 5:48 PM, P. Oscar Boykin
9/18/16
ravi kiran holur vijay
,
Oscar Boykin
7
9/11/16
Strange (or inconsistent) behaviour for GroupBy -> SortBy
Just to be sure can you try with scalding 0.16.0? On Sun, Sep 11, 2016 at 11:23 ravi kiran holur
unread,
Strange (or inconsistent) behaviour for GroupBy -> SortBy
Just to be sure can you try with scalding 0.16.0? On Sun, Sep 11, 2016 at 11:23 ravi kiran holur
9/11/16
Kostya Salomatin
, …
Alex Levenson
3
8/31/16
Flow optimization question.
I think you *can* tune the min/max size by using the sourceConfInit method in Sources and applying
unread,
Flow optimization question.
I think you *can* tune the min/max size by using the sourceConfInit method in Sources and applying
8/31/16
Xiaolin Li
, …
Oscar Boykin
4
8/26/16
taking field names as argument
yes, you can move over bit, by bit. To go from a `Pipe` to `TypedPipe` you need to use `TypedPipe.
unread,
taking field names as argument
yes, you can move over bit, by bit. To go from a `Pipe` to `TypedPipe` you need to use `TypedPipe.
8/26/16
Kostya Salomatin
,
P. Oscar Boykin
3
8/24/16
Strange java heap space in mapper, solved by .forceToReducers
Thanks, good to know. On Wednesday, August 24, 2016 at 12:52:23 AM UTC-7, P. Oscar Boykin wrote: Yes,
unread,
Strange java heap space in mapper, solved by .forceToReducers
Thanks, good to know. On Wednesday, August 24, 2016 at 12:52:23 AM UTC-7, P. Oscar Boykin wrote: Yes,
8/24/16