Nested block

12 views
Skip to first unread message

Tong Li Ken

unread,
Jun 13, 2016, 8:43:54 AM6/13/16
to treehugger-scala
Hi, 

I am using treehugger to generate spark streaming code and i have some issue to generate this block of codes.
Can anyone help?

dstream.foreachRDD { (rdd, time) =>
  rdd.foreachPartition { partitionIterator =>
    val partitionId = TaskContext.get.partitionId()
    val uniqueId = generateUniqueId(time.milliseconds, partitionId)
    // use this uniqueId to transactionally commit the data in partitionIterator
  }
}
Reply all
Reply to author
Forward
0 new messages