Blog post: turning Scala code into Spark

[Anthony Cros]

Hi all,

Hopefully it is ok for me to post this here, I just finished writing a blog post entitled "Turning Scala code into Spark" at http://anthonycros.com/turn-scala-into-spark/ (also on github at https://anthony-cros.github.io/blog/turn-scala-into-spark/). I would love to have some feedback from the community!

I will link comments on here to the dedicated issue I created for discussing it: https://github.com/anthony-cros/anthony-cros.github.io/issues/1 (low tech, but I haven't set everything up for that blog yet).

Thanks a lot in advance!

Anthony

[Koert Kuipers]

interesting!

note that spark's flatMap is a bit of a challenge in the context of keeping lines the same verbatim, since its RDD[X].flatMap[Y](f: X => TraversableOnce[Y]) instead of RDD[X].flatMap[Y](f: X => RDD[Y])

--
You received this message because you are subscribed to the Google Groups "scala-debate" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-debate+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Anthony Cros]

Thanks for the feedback, Koert!

I think in the future I'll try and use a more comprehensive example, and this time I'll include a flatMap. I think it's worth pointing out were Scala and Spark do differ as well. This would be valuable info for someone having to do such a translation (such as myself soon actually!). I'd even like to create a cheatsheet for it eventually...