Unable to launch basic Dataproc cluster

[Moses Sundheep]

HI Team,

I am creating basic dataproc cluster with 1 node, but it is failing to create the same.

Earlier i am able to create the same without errors:

Error:

For PD-Standard without local SSDs, we strongly recommend provisioning 1TB or larger to ensure consistently high I/O performance. See https://cloud.google.com/compute/docs/disks/performance for information on disk I/O performance failed

Thanks & Regards,

Moses

[karth...@google.com]

Can you please add the full command and full output -- the note about disk size is a just a warning. There is likely a different underlying error.

Feel free to remove your project id when you paste the command/output.

[Moses Sundheep]

Thank you Karthik for your response, 

{

  "protoPayload": {

    "@type": "type.googleapis.com/google.cloud.audit.AuditLog",

    "status": {

      "code": 7,

      "message": "PERMISSION_DENIED"

    },

    "authenticationInfo": {

      "principalEmail": "******@gmail.com"

    },

    "requestMetadata": {

      "requestAttributes": {},

      "destinationAttributes": {}

    },

    "serviceName": "dataproc.googleapis.com",

    "methodName": "google.cloud.dataproc.v1beta2.ClusterController.CreateCluster",

    "resourceName": "projects/project-267608/regions/us-central1/clusters/cluster-2b88"

  },

  "insertId": "-w6ctkoc4ek",

  "resource": {

    "type": "cloud_dataproc_cluster",

    "labels": {

      "project_id": "project-267608",

      "region": "us-central1",

      "cluster_name": "cluster-2b88",

      "cluster_uuid": "3cf1eb4c-fd97-4abf-9951-2e8f33a7d568"

    }

  },

  "timestamp": "2020-03-05T00:49:31.807Z",

  "severity": "ERROR",

  "logName": "projects/spark-project-267608/logs/cloudaudit.googleapis.com%2Factivity",

  "operation": {

    "id": "projects/spark-project-267608/regions/us-central1/operations/05bfc3b1-d146-49c3-98c8-5b7d6442bbc3",

    "producer": "dataproc.googleapis.com",

    "last": true

  },

  "receiveTimestamp": "2020-03-05T00:49:31.877478364Z"

}

[karth...@google.com]

Ah it seems to be an issue with billing. The operation should have more information if you run `gcloud dataproc operations describe 05bfc3b1-d146-49c3-98c8-5b7d6442bbc3`.

[Moses Sundheep]

Thank you Karthik, its issue with the billing. 

Once again thank you for your help.

[Vishal M]

Hi Team,

I am creating a dataproc cluster for learning purpose, facing the following errors, can anyone please help me out.

• • For PD-Standard without local SSDs, we strongly recommend provisioning 1TB or larger to ensure consistently high I/O performance. See https://cloud.google.com/compute/docs/disks/performance for information on disk I/O performance.
  • The firewall rules for specified network or subnetwork would allow ingress traffic from 0.0.0.0/0, which could be a security risk.
    
    Thanks

[Richard Holowczak]

Hi Vishal

Those two messages are warning and not errors.

Your cluster should have been created. If not, you would see error messages in a pop up window.

The default cluster uses regular hard disks (not SSDs) so the first message suggests using larger storage in order to get better performance. This is not a big deal if you are just learning Dataproc.

The second message shows up when you select the Component Gateway. It warns you that the firewall rule basically lets in connections from any source IP address.  This is OK for learning purposes because you don't have to fuss with opening up the firewall for your specific IP address.

Obviously if this were a production cluster you would close up the firewall to pretty much everything.

Cheers

Rich H.