Trying to use Google Cluster data set

[SHREYA PANDEY]

Hello, 

My name is Shreya Pandey, and I'm currently a senior majoring in ECE. For a class project, my team and I are trying to recreate a paper -- that uses this clusterdata-2019 dataset. We want to get scheduling data, and so we're hoping to download thisHello, 

My name is Shreya Pandey, and I'm currently a senior majoring in ECE. For a class project, my team and I are trying to recreate a paper -- that uses this clusterdata-2019 dataset. We want to get scheduling data, and so we're hoping to download this data into perlmutter and train a reinforcement learning model. However, when I use google cloud in order to query the data from here, I get " User does not have bigquery.jobs.create permission in project google.com:google-cluster-data"

Can I please have some guidance on how to retrieve the trace data?

Best, 

Shreya P data into perlmutter and train a reinforcement learning model. However, when I use google cloud in order to query the data from here, I get " User does not have bigquery.jobs.create permission in project google.com:google-cluster-data"

Can I please have some guidance on how to retrieve the trace data?

Best, 

Shreya P

[SHREYA PANDEY]

For reference, this is the paper I'm trying to recreate: Trying to use Google Cluster data set

This was the code I was trying to query the data: 

from google.cloud import bigquery

project_id = 'google.com:google-cluster-data'

client=bigquery.Client(project=project_id)

query = """

SELECT capacity.cpus AS cpu_cap,

capacity.memory AS memory_cap,

COUNT(DISTINCT machine_id) AS num_machines

FROM `google.com:google-cluster-data.clusterdata_2019_a.machine_events`

GROUP BY cpu_cap, memory_cap

"""

# running this query:

query_job = client.query(query)

results = query_job.result().to_dataframe()

print(results)

[Sagar Chauhan]

Will check tomorrow

--
You received this message because you are subscribed to the "Google cluster data - discussions" group. To post to this group, send email to googlecluste...@googlegroups.com. To unsubscribe from this group, send email to googleclusterdata-...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/googleclusterdata-discuss?hl=en-US.
---
You received this message because you are subscribed to the Google Groups "Google cluster data - discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to googleclusterdata-...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/googleclusterdata-discuss/c2e527ea-8013-4197-a7ac-3b359b23c60fn%40googlegroups.com.

[john wilkes]

You are trying to run the queries using the google.com:google-cluster-data projects' resources.  You need to run them in (and charge them to) your own project.

  john

To view this discussion visit https://groups.google.com/d/msgid/googleclusterdata-discuss/CADmVS2_PXRcsQvH0Y0e28%3Dxi9w%2BhjpsJT3PKMj_KD2VhVEOHbw%40mail.gmail.com.

[Abrar Hossain]

Hi Shreya,

Can't see the paper referenced. Can you send it again?

--

You received this message because you are subscribed to the "Google cluster data - discussions" group. To post to this group, send email to googlecluste...@googlegroups.com. To unsubscribe from this group, send email to googleclusterdata-...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/googleclusterdata-discuss?hl=en-US.
---
You received this message because you are subscribed to the Google Groups "Google cluster data - discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to googleclusterdata-...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/googleclusterdata-discuss/c2e527ea-8013-4197-a7ac-3b359b23c60fn%40googlegroups.com.

--

___________________________

Best Regards,

Abrar Hossain

Contact Number: (+1) 419.320.7896