Azure Retrying after 60 seconds

[badappleuk]

Hey Team,

I've been able to connect and it looks like it's pulling back data, however it's throwing this error often :

Azure ConsumptionManagementClient.usageDetails.listNext failed. Reason: Too many requests. Please retry after 60 seconds.

Is there anyway to slow down the requests so I don't bump into this error?

Thanks

[Dan Lewis-Toakley]

Hi Daniel,

Thanks for reaching out -- this has been a common topic/question the past few weeks! 

We have implemented back-off and retry logic based on the response from Azure's API - in this case, 60 seconds. When you see the error logged, it should just be a warning, with another log write after saying it will be retried. Like this:

2021-12-08T20:22:45.664Z [ConsumptionManagement] warn: Azure ConsumptionManagementClient.usageDetails.listNext failed. Reason: Too many requests. Please retry after 60 seconds.
2021-12-08T20:22:45.668Z [ConsumptionManagement] info: Retrying after 60 second

If that is the case, then it should eventually succeed, even though it might be logging a lot of warning messages. 

However, if you would like to reduce the number of warnings and potentially the number of back-off retries, you could instead make requests for subscription data in chunks/groups. I've created this PR if you want to pull the branch azure-subscription-chunks and try using that instead. It's chunking in groups of 10 which seemed to work well for us, but feel free to try other options too.  

Let us know how it goes. If it's more performant for you and reduces the warnings, that is a good case to make chunking/grouping the subscriptions the default approach.

Best,

Dan 

--
We're excited to hear from you and hope you're enjoying Cloud Carbon Footprint.
Please fill out our feedback form: https://forms.gle/nFzkRioryy4R1DGB6
Add your name to ADOPTERS.md: https://github.com/cloud-carbon-footprint/cloud-carbon-footprint/blob/trunk/ADOPTERS.md
Give us a star on the github if you're enjoying the tool!
---
You received this message because you are subscribed to the Google Groups "Cloud Carbon Footprint" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloud-carbon-foot...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-carbon-footprint/5967e7a6-a715-4e5d-abae-eada086bae9fn%40googlegroups.com.

--

Dan Lewis-Toakley Green Cloud Lead, North America Pronouns  He/Him Email  dan.lewi...@thoughtworks.com Telephone  +19172545068

[badappleuk]

Thanks Dan! 

Will give it a try now and report back! 

Question about recommendations for Azure, are they only for AWS and GCP in the current builds?

[Dan Lewis-Toakley]

Another good question -- but yes, unfortunately we only support recommendations for AWS and GCP right now, but have it on the roadmap to support Azure as well. We had some blockers on implementing recommendations for Azure and have reached out to their support team for help/guidance. 

To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-carbon-footprint/82bc1077-4061-4d06-800f-a91ebcf6646bn%40googlegroups.com.