Q on Spark Streaming: When do we know that all input has been consumed?

[sku...@lucirix.com]

My current spark streaming application is not fine-tuned, i.e. I think Spark streaming is falling behind the input.

Assuming no more input after a certain point, how do I determine that Spark is done?

Right now, I am writing out my resulting RDD to hdfs and when I notice no size increases, I am assuming that Spark is done.

There must be a better way :)

cheers

Kumar