How to get the number of rows/records in a spark RDD?
864 views
Skip to first unread message
Gaurav Dasgupta
unread,
Oct 11, 2012, 3:10:17 PM10/11/12
Reply to author
Sign in to reply to author
Forward
Sign in to forward
Delete
You do not have permission to delete messages in this group
Copy link
Report message
Show original message
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to spark...@googlegroups.com
HI,
I am a beginner in Spark. Is there any way that I can get the total number of rows/records from a spark RDD/Input text file?
Actually I am trying to run an iterative mapreduce job and it should iterate for the total number of rows in the input text file. Is there any way for this?
Thanks, Gaurav Dasgupta
Reynold Xin
unread,
Oct 11, 2012, 3:12:55 PM10/11/12
Reply to author
Sign in to reply to author
Forward
Sign in to forward
Delete
You do not have permission to delete messages in this group
Copy link
Report message
Show original message
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message