flatMapped

38 views
Skip to first unread message

Alex Cozzi

unread,
Apr 5, 2013, 11:40:07 AM4/5/13
to scoob...@googlegroups.com
Playing with 0.7 I noticed that flatMap is deprecated and replaced by flatMapped. I think I see the point made in the deprecation message, but surely it does look ugly...

Eric Torreborre

unread,
Apr 7, 2013, 8:20:45 AM4/7/13
to scoob...@googlegroups.com
Yes, it is "mapFlatten" and not so gorgeous. If you have a better idea...

Alex Cozzi

unread,
Apr 8, 2013, 5:43:21 PM4/8/13
to scoob...@googlegroups.com
I think probably I am not appreciating the reason for the change. 

The reason why I like the current flatMap is that in general I like to think of DLists just as collection classes. I actually started to write programs using Seq running locally and playing with .par, and then when I want to go with really large datasets I just replace the collection creation with Scoobi's code reading from HDFS, and I never had issues with it. If now I will have to replace all the flatMap with mapFlatten it kinds of break this model. 

I checked the signature of flatmap in GenTraversableLike:

def flatMap[B](f: A => GenTraversableOnce[B]): $Coll[B]

your arguments goes that DList is not a GenTraversableOnce. I see that, but what trouble does it cause to say that DList API is inspired by the collection APIs instead of being identical to it ? 



Christopher Severs

unread,
Apr 8, 2013, 8:24:19 PM4/8/13
to scoob...@googlegroups.com
I wanted to second what Alex says. One of the biggest advantages of Scoobi is the ability to try things out locally in the repl on scala collections and then essentially just change the input and output and have it run on our Hadoop cluster. Changing the API away from the core collection operations makes that use case more difficult and I don't see what the gain is.

Eric Torreborre

unread,
Apr 8, 2013, 9:54:51 PM4/8/13
to scoob...@googlegroups.com
We still hope to be able to implement a "real" flatMap in the future. I'm sure if it's really possible but if it is we'd like to reserve that name for this correct operation. 

It then opens lots of possibilities, we can use the for notation and use library functions supporting Monads.

E.
Reply all
Reply to author
Forward
0 new messages