How can we print some output in the map function

[Xiaozhe Wang]

Hi all,

I am debugging an application on pyspark. When I call 

rdd.map(f).collect()

It will throw some errors. There should be some problems in the function f. I try to print something out and write something to the  file system in the function f, but it cannot work.

Can anybody tell me how to print something out for debugging the application?

Thanks,

Kaden

[Cheng Lian]

Hi Kaden.  Are you running your application on a Spark cluster?  If so, `f' is executed by the executor process on the worker node, thus you should look for your debugging output file on the worker node.