Dataproc job duration metric issue

[rushi rushi]

Hi,

I'm working dataproc cluster and I need monitor how long jobs are running, for that I'm using "dataproc.googleapis.com/cluster/job/completion_time". But results of this metric is not matching with the actual time jobs are taking.

Below in the screenshots you can see that I'm running two jobs which taking 10 mins each.

But in chart total completion time is 17 min.

Please Help

[Srikanth Potukuchi]

Hi, 

I would suggest looking at "Life of a Dataproc Job": https://cloud.google.com/dataproc/docs/concepts/jobs/life-of-a-job 

and also Access job logs in logging:

https://cloud.google.com/dataproc/docs/guides/logging#accessing_job_logs_in

Regards,

Srikanth