Input dataset:1,2008-10-23 16:05:05.0,\N,Donald,Becton,2275 Washburn Street,Oakland,CA,94660,5100032418,2014-03-18 13:29:47.0,2014-03-18 13:29:47.0
2,2008-11-12 03:00:01.0,\N,Donna,Jones,3885 Elliott Street,San Francisco,CA,94171,4150835799,2014-03-18 13:29:47.0,2014-03-18 13:29:47.0
3,2008-12-21 09:19:50.0,\N,Dorthy,Chalmers,4073 Whaley Lane,San Mateo,CA,94479,
6506877757,2014-03-18 13:29:47.0,2014-03-18 13:29:47.0
Using keyBy I have to create an RDD with the postal code(9th field) as the key which I did like belowval keydbycode = rdd.keyBy(line => line.split(',')(8))
res3: Array[(String, String)] = Array((94660,1,2008-10-23 16:05:05.0,\N,Donald,Becton,2275 Washburn Street,Oakland,CA,94660,5100032418,2014-03-18 13:29:47.0,2014-03-18 13:29:47.0), (94171,2,2008-11-12 03:00:01.0,\N,Donna,Jones,3885 Elliott Street,San Francisco,CA,94171,4150835799,2014-03-18 13:29:47.0,2014-03-18 13:29:47.0), (94479,3,2008-12-21 09:19:50.0,\N,Dorthy,Chalmers,4073 Whaley Lane,San Mateo,CA,94479,6506877757,2014-03-18 13:29:47.0,2014-03-18 13:29:47.0))Now using mapValues I have to create a pair RDD with postal code as the key and a list of names (Last Name,First Name) in that postal code as the value I triedbelow which results in
error: (String, Array[String]) does not take parameterskeydbycode.mapValues(line => line.split(',')).map(fields => (fields(3), fields(4)))
scala> keydbycode.mapValues(line => line.split(',')).map(fields => (fields(3), fields(4)))
<console>:28: error: (String, Array[String]) does not take parameters
keydbycode.mapValues(line => line.split(',')).map(fields => (fields(3), fields(4)))
^
<console>:28: error: (String, Array[String]) does not take parameters
keydbycode.mapValues(line => line.split(',')).map(fields => (fields(3), fields(4)))
^
Could someone explain me how I can get this done using mapValues ?
Thanks in advance !!