How to support other cloud providers - smallest possible first step

[Chris Adams]

Hi folks.

Can I have a bit of help figuring out how to make the simplest possible starter for cloud providers who are not Azure, AWS or GCP ?

A summary of how cloud carbon footprint works

This obviously leaves out loads of details that are explained in the methodology, but for the purposes of this discussion, I'm writing this to sanity check my thinking, for a single provider offering a single service as a starting point.

Starting with Compute

The cloud carbon footprint docs themselves say the general approach is as follows:

(Cloud provider service usage) x (Cloud energy conversion factors [kWh]) x (Cloud provider Power Usage Effectiveness (PUE)) x (grid emissions factors [metric tons CO2e])

I assume cloud service provider usage here would be something like:

• 100 hours of a XXL EC instance
• some equivalent from another provider, like  a 100 hrs of a 32gb droplet from digital ocean, or
• 24hrs of a 16gb instance at $SOME_CLOUD_PROVIDER.

This is basically a quantity. You could start with a single item from a single new provider.

Cloud energy conversion factors [kWh]

From these you apply the conversation factor to turn usage into an amount of energy used by the servers.

You can see some sample figures from various providers in this issue on scaphandre:

https://github.com/hubblo-org/scaphandre/issues/25

And if we look at these you'll see this is pretty much what powers cloud carbon footprint's own numbers for the big providers's compute instance numbers, in the form of these carbon coefficients.

https://github.com/cloud-carbon-footprint/cloud-carbon-coefficients/blob/main/coefficients.ipynb

As I understand it, there are some assumptions about utilisation for hyperscalers which tend to be higher than others, but otherwise, if we know the kinds and qauntities of CPUs and RAM used inside, we can follow an approach which is not wildly inconsistent with how it's done for the bigger tech providers.

Cloud provider Power Usage Effectiveness (PUE)

Once you have this, you apply the PUE for the non-compute bits.

This basically makes your number slightly larger, because all the providers in cloud carbon footprint right now are hovering around 1.1 to 1.2.

It would likely be higher in smaller providers as they don't have the same colossal CapEx investment in huge datacentres.

grid emissions factors [metric tons CO2e]

…and then finally the carbon intensity of the energy used to power everything, to turn it into carbon.

Making this for a single provider

If you know the info above for a machine you're using from a provider, I think  you could add support for them, but I'm a little unclear on the time resolution you'd need to consume this information in the charts.

If it helps, I created this issue on the cloud carbon footprint repo, where I asked some similar questions and provided some more context. I guess you could add a reaction there to express interest, as I'd like to approach some providers to talk about implementing support.

https://github.com/cloud-carbon-footprint/cloud-carbon-footprint/issues/512

[Danielle Erickson]

Hey Chris,

Sorry for the delay and thanks for spelling all this out!

From a high level, your explanation is a good summary of how CCF calculates carbon.

We'd love the addition of support for other providers, and as you said in your issue, basically if we're able to gather the proper usage data from the cloud provider, it should be possible to build support for it into the tool. We'll respond to your ticket with the data that would be necessary. There would be some additional work to classify usage types for the various services, estimate replication factors, map regions with grid emissions factors, etc. but identifying if we have the data available would be the first step.

Hope this helps get things started,

Cheers,

Danielle

--
You received this message because you are subscribed to the Google Groups "Cloud Carbon Footprint" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloud-carbon-foot...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-carbon-footprint/6706c222-f093-4e09-be05-a83e2a82e337n%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Danielle EricksonGreen Cloud Product Manager and Senior Consultant Developer

Email

deri...@thoughtworks.com

Telephone

609 423 8201

[Chris Adams]

OK,  thanks Danielle,

It does look like you can follow a similar approach to the utilization based methodology used for AWS EC2 instances, and apply a similar approach with any provider that exposes CPU usage metrics, and the number of CPUS for a machine, at an hourly basis.

I've fleshed out issue 512 in more detail to outline how I think it could be done with Hetzner (a larger German provider), but I think the same approach would work for Digital Ocean Droplets too, as they also expose CPU utilisation at an hourly level.

I'll post updates to that issue until it makes sense to create a newer, better specified ticket/issue for a specific provider