Unable to find Yarn Aggregated and spark driver logs in Cloud Logging

[Shivam Soliya]

Hey people,

I am following this doc while creating a new dataproc cluster. And I have set dataproc properties for spark driver logs to be 1.cluster-2.true-3.true and I am expecting dataproc.jobmyarn.container logs in dataproc job resource. But I am not getting any dataproc job logs. Is there any reason for that? How can I get the job logs? I need yarn memory seconds per job and vcore seconds per job in logging and metrics dashboard as well. Help would be highly appreciated.

Thank you.