Limited BigQuery access re-enabled

[M-Lab Team]

Hi everyone, 

We have re-enabled access to Measurement Lab’s public BigQuery for limited access. More specifically, each user is now limited to 2 TiB daily worth of “bytes processed” for their total queries. This limit will remain in place until the end of April, at which point we’ll reassess what quotas make sense long-term. 

Until then, to ensure that we can support data usage for as many researchers as possible, please be aware of the amount of bytes your queries are processing. You can see how many bytes your query will process by doing a query “dry run” [documentation]. Please also be sure to use date partitions at all times.

We are re-enabling access with an interest in supporting researchers but are unfortunately still recovering from the resource utilization incident that took place on 27 April. Through the end of the month, we will need to monitor our capabilities daily. Please be advised that we might need to temporarily suspend access again, though we will do our best to avoid it. 

Thank you for your cooperation and attention. Feel free to reach out to us at sup...@measurementlab.net if you have any questions. 

Best,

M-Lab Team

[M-Lab Team]

Edit: The incident took place on 27 March. 

[Ziv Weissman]

Does this include doing BigQuery API calls through python?

I've been trying to make some calls to download some pcaps today but I've been getting problems with my permissions.

Certain calls that are working well for me through big query are not working when I try to do them through python today.

Thanks,

Ziv

[Lai Yi Ohlsen]

Hey Ziv, 

We've restored permissions to the same state that they were before the incident, so you should be able to do everything you could before. Could you share your queries and the permissions error you're getting? 

--
You received this message because you are subscribed to the Google Groups "Measurement Lab Team" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hello+un...@measurementlab.net.
To view this discussion on the web visit https://groups.google.com/a/measurementlab.net/d/msgid/hello/483b16bf-a7da-4bf4-a9d2-a97a7e712711n%40measurementlab.net.

--

Lai Yi Ohlsen

Lead Data Scientist

Measurement Lab

Code for Science & Society