Using CCF Methodology to calculate footprints by hand

[Elie]

Hello, 

I am considering using the ccf calculator for professional use. But I was curious to estimate my current company CO2 emissions on private and on-prem DCs. So, I tried to manually calculate our emissions with your methodology. But the results I got are well under what I expected. 

You will find below the calculation I applied to my data. I believe I made a mistake somewhere, but I can’t find it. 

For simplicity sake, I will not detail the different coefficients, but I applied different coefficients depending on several factors such as DC's size, location and other parameters such as PUE. 

Calculation:

Total CPU Calculation in Wh:

= Total number of CPU * Number of Hours in Year * Average CPU Watts

With:
Average CPU Utilization = 0,5

Number of Hours in Year = 8760

Average CPU Watts = 2,27

Total Storage Calculation in Wh:

= Total SSD in GB * (SDD energy estimation (Wh/TB)/1000) + Total HDD in GB * (HDD energy estimation (Wh/TB)/1000)*Number of Hours in Year

With: 

SSD energy estimation (Wh/TB) = 1.2 

Number of Hours in Year = 8760

Total Memory Calculation in Wh:

= Total RAM in GB * Memory Watts per GB * Number of Hours in Year

With: 

Memory Watts per GB = 0.392

Number of Hours in Year = 8760

Total Wh Calculation: 

= Total CPU Calculation in Wh + Total Storage Calculation in Wh + Total Memory Calculation in Wh

Total Carbon Emission Calculation:

=(Total Wh/1000) * Grid emission factor (tCO2/kwh)

My calculations miss some elements and other elements (as RAM) will be overvalued, but the result I get from this calculation is very low compared to what I expected. 

Do you think this calculation is correct or did I miss something? 

Sincerely, 

Elie

[Cloud Carbon Footprint]

Hi Elie,

Thanks for reaching out!

 

We have not completed work to calculate on-prem emissions just yet so we are very excited to get this going and have been looking to work with a contributor to configure a good methodology!

Would you be interested in setting up a call to discuss in more detail? In the meantime, I have provided some notes:

 

Compute:

 Our Methodology:

 Kilowatt hours =  ((minWatts + (averageCPUUtilization / 100) * (maxWatts - minWatts)) *

   virtualCPUHours *

   powerUsageEffectiveness) /

1000 // convert to kWh

 ((minWatts + (averageCPUUtilization / 100) * (maxWatts - minWatts):  I am thinking your value of 2.27 is too low. With our methodology, we are working with virtual CPU's so we are dividing the min/max watts by the number of threads. Since I believe you are working with physical CPU's rather than virtual, you should not need to divide by the threads. Instead, you should use the average across the entire SPECPower DB, or if you know what microarchitecture you are using, take the average of that subset. You can even go even more granular if you know exactly which processors are used.

Here you can reference the SPECPower Database: https://github.com/cloud-carbon-footprint/cloud-carbon-coefficients/blob/main/data/SPECpower-full-results.csv

I'm thinking your min/max watts should ideally be at least 100x higher so hopefully this makes sense.

virtualCPUHours: in your case i'm thinking this may be 24 hours (per day) * number of physical server chips which could be determined from your underlying micro-architecture or looked up in the SpecPower DB

PUE: this would need to be determined probably somewhere in the range of 1.2 - 1.6

Storage:

Our Methodology:

 Kilowatt hours = (terabyteHours *

  this.coefficient *

  powerUsageEffectiveness /

1000 // convert to kWh

 

I'm thinking here you may just need to convert your total SSD/HDD in GB to TB Hours based on hours per day which would be:  (usageAmount / 1000) * 24

Then you would use the correct coefficient based on SDD: 1.2 or HDD: .65

as well as the determined PUE

 

Memory:

 Our Methodology:

 Kilowatt hours = Memory usage (GB-Hours) x Memory coefficient

 

I think you have this correct at a high level, I would again just use hours in a day 

 

Hope this helps!

 

Thanks,

Cam

[Elie]

Hi CCF,

Thank you for your answer!

On compute:

You are right, I recalculated my results with the ratio of min & max watts/chips, and the results look way more coherent.

For service providers, Average CPU Wh = 65

For local DC, Average CPU Wh = 53

Unfortunately, I don’t have access to the type of micro-architecture.

For memory:

I have also adjusted my calculations to avoid overestimating the RAM energy consumption. Using the following Excel formula:

IF(Sum of RAM/Sum of Chips> Average Ratio RAM/Chips; Sum of RAM-(Sum of Chips* Average Ratio RAM/Chips);0)

Average Ratio RAM/Chips = 51 (source: SPEC DB)

 

Concerning meeting to discuss a methodology, I would be happy to discuss this issue in detail and help develop a method for on-prem clouds. But I first have to get approval from my hierarchy. Hence, I will get back to you soon with an answer.

For the time being, I would prefer to keep our discussion private, of course, you can republish your insights and mines (anonymously). 

Sincerely yours,

Elie 

[Julian Len]

Hi!

I'm also interested in calculating consumption by hand. 

On Compute, watching at the code there is a "replication factor". Why did not you get it into account?

Thanks!

[Cloud Carbon Footprint]

Hello!

We are very open to any suggestions as we look to implement an approach for calculating consumption by hand.

In Elie's case, I made the assumption that there are no back up servers which is where we would consider a replication factor. In the cloud, there is the concept of having redundancy with multiple availability zones to make your cloud process fail-proof. Depending on the storage service, we have varying replication factors. You can view how we determine those here:
https://www.cloudcarbonfootprint.org/docs/methodology#storage-services-replication-factors

Thanks,

Cam

[Elie]

Hello Cam, Jul,

Indeed, I took into account the replication factor directly in the amount of storage I provisioned. 

Sincerely, 

Elie

[Cloud Carbon Footprint]

Hi Elie, Jul,

Our team is going to plan a concerted effort towards calculating on-premise footprints. Would you both be interested to setting up a call to discuss ways to collaborate in more detail? We would be happy to meet this week or next!

Thanks,
Cam

[Elie]

Hi Cam,

Let me come back to you by tomorrow! 

Sincerely, 

Elie

--
We're excited to hear from you and hope you're enjoying Cloud Carbon Footprint.
Please fill out our feedback form: https://forms.gle/nFzkRioryy4R1DGB6
Add your name to ADOPTERS.md: https://github.com/cloud-carbon-footprint/cloud-carbon-footprint/blob/trunk/ADOPTERS.md
Give us a star on the github if you're enjoying the tool!
---
You received this message because you are subscribed to a topic in the Google Groups "Cloud Carbon Footprint" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cloud-carbon-footprint/rqvHCFjw1Fk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to cloud-carbon-foot...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-carbon-footprint/898863ae-30a5-4904-936b-b3f9d76520b4n%40googlegroups.com.

[Elie]

Hi Cam, 

As promised, I am getting back to you. And I am pleased to tell you that our answer is positive. 

Also, I am currently working with Accenture on the subject and would like to invite them as well if you agree. 

We can discuss the date of the meeting by email, with: 

- my professional email: elie.de...@loreal.com 

- and the email of the Accenture consultant: martin...@accenture.com

Sincerely yours, 

Elie de Perthuis

[Elie]

Hello Cam, 

I get back to you about the meeting on the method to assess the carbon footprint of on-prem DC. 

Indeed, here in France most of us are soon (17/12) in Christmas holiday. Hence, it would be preferable to meet before the end of the week. If not possible, I personally am available until 31/12 

Sincerely yours, 

Elie 

PS: We would be glad to share information with you, I strongly believe it would be mutually beneficial! 

[Julian Len]

Hey everyone!

Hope you are doing well.

Why aren't you using 'power usage effectiveness' in the memory consumption calculation?

Sincerely yours,

Juli

[Cloud Carbon Footprint]

Hi Juli,

We are factoring in PUE for memory estimations. See here:
https://github.com/cloud-carbon-footprint/cloud-carbon-footprint/blob/50bd22104dcb42a20bc4314ca3d7e6403aff4a95/packages/core/src/memory/MemoryEstimator.ts#L55

I had just left it out above earlier in the thread.

Thanks,

[Julian Len]

Thanks for the quick response!

I meant when the formula to calculate the CCF for the memory was presented, it was

Memory:

 Our Methodology:

 Kilowatt hours = Memory usage (GB-Hours) x Memory coefficient

Which I think the PUE was forgotten.

Finally today I've arrived at a number for each category, I'll post it above later

Thank you!

Juli

[Julian Len]

Hi everyone!

My apologies for the number of messages, I hope to be closer!

Finally, I arrived at a number. This is the procedure I followed:

The company I worked at provided me the next information regarding the AWS instance:

Instance family and type: EC2 m5a.large

Processor: AMD_EPYC_1ST_GEN

Storage used: 230 GB SSD (not the total allocated)

Region: us-west-1-North California.

In all the cases, I assumed a replication factor of 1.

Compute

Average Watts = ((minWatts + (averageCPUUtilization / 100) * (maxWatts - minWatts))

Compute Watt-Hours = Average Watts * vCPU Hours

Kilowatt hours [kWh] =   Compute Watt-Hours *

   powerUsageEffectiveness) / 1000 // convert to kWh

minWatts[AMD_EPYC_1ST_GEN] = 0.82

maxWatts[AMD_EPYC_1ST_GEN]: 2.55

averageCPUUtilization: 50,

powerUsageEffectiveness: 1.135

Regarding vCPU hours I used 24hs. Is it correct? Since it is an instance working 24 hours a day.

Storage

We are using an SSD

Kilowatt hours = terabyteHours *  this.coefficient *  powerUsageEffectiveness /1000 // convert to kWh

terabyteHours = Our instance has occupied 230GB, do I have to calculate totalOccupied*24h?

this.coefficient = 1.2 Watt-hours (SSD)

powerUsageEffectiveness = 1.135

Memory

Kilowatt hours = [Memory (GB) exceeding SPECPower database average] x [Memory coefficient] x usage amount (Hours) * powerUsageEffectiveness

 [Memory (GB) exceeding SPECPower database average] = 6.13 (I did the calculation using the family of the AWS instance: m5a)

[Memory coefficient] = 0.000392, // kWh / Gb

powerUsageEffectiveness = 1.135

usage amount: Regarding usage amount, I am a bit lost here. Do I need to know how much memory has my instance? Or just use "24hours"?

Networking

Kilowatt hours: gigaBytes * this.coefficient * powerUsageEffectiveness

gigaBytes = The only information I have is NetworkIn and NetworkOut. 

Network In is 108GB while NetworkOut is 53.8GB. What should I put?  AFAIK, this is the total usage of the instance. So, should I use the total usage of the instance but in a day? 

this.coefficient =  0.001, // kWh / Gb

powerUsageEffectiveness = 1.135

When the "emission factor" is calculated, what is the resulting unit? 

Do you think it is correct? I followed your answer here and the codebase at the same time. I wrote the questions in red so it is easier to see.

Thank you very much!

Greetings.

Juli

[James Hall]

I've been trying something similar and this thread has helped a lot but when I work out the computation carbon it comes out very low considering in the demo graph it says that 54 KWhs of one AWS account using EC2 will release 3 metric tonnes of CO2 into the atmosphere

Min watts        107.414286
Max watts        331.985714
averageCPUUtilization        50%
CPU hours        200
PUE        1.2
Average watts        219.7
Compute Watt-Hours        43940
Kilowatt hours [kWh]         52.728
GEF of england (tonnes)        0.000235
CO2e (tonnes)        0.01239108

Any assistance in where I'm going wrong would be helpful. I've seen in this thread that the chips need to be taken into account but it doesn't appear to mention it in any equations in the methodology page

[Oran Maliphant]

Just wondered if there was any methodology that incorporates GPU power usage as well as CPU/Memory/Disk ?
our org has heavy GPU users as well as 3rd party hardware that are developed for specific number crunching tasks.

has anyone solved for this yet?

[Dan Lewis-Toakley]

Hi Oran -- apologies for the delayed reply, but wanted to answer your question so the broader group can read it! 

Right now we don't support GPUs, but it is one of our priorities at the moment and we are actively working on a methodology for it. We will update the group once we have published something!  

You received this message because you are subscribed to the Google Groups "Cloud Carbon Footprint" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloud-carbon-foot...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-carbon-footprint/91d779e2-0df4-497a-9516-0c5681ccde5fn%40googlegroups.com.

--

Dan Lewis-Toakley Green Cloud Lead, North America Pronouns  He/Him Email  dan.lewi...@thoughtworks.com Telephone  +19172545068

[Julian Len]

Hi there!

I'm done with the numbers. Sending again the procedure.

TL:DR: The total consumptions gave me 3,38 co2eq, how do you see this number? Is it reasonable? 

Compute

Average Watts = ((minWatts + (averageCPUUtilization / 100) * (maxWatts - minWatts))

Compute Watt-Hours = Average Watts * vCPU Hours

Kilowatt hours [kWh] =   Compute Watt-Hours *

   powerUsageEffectiveness) / 1000 // convert to kWh

minWatts[AMD_EPYC_1ST_GEN] = 0.82

maxWatts[AMD_EPYC_1ST_GEN]: 2.55

averageCPUUtilization: 50,

powerUsageEffectiveness: 1.135

Regarding vCPU hours = 24hs

(( 0.82 + (50/100) * (2.55 - 0.82)) * 24hs * 1.135 / 1000

((0.82 + 0,5 * 1,73) * 24hs * 1.135 / 1000

1,685 * 24 * 1,135 / 1000

45,9 / 1000 = 0,0459

Storage

We are using an SSD

Kilowatt hours = (terabyteHours *  this.coefficient *  powerUsageEffectiveness) /1000 // convert to kWh

terabyteHours = 100GB/1000

this.coefficient = 1.2 Watt-hours (SSD)

powerUsageEffectiveness = 1.135

((100GB/1000)TB * 24 * 1.2 * 1,135)/1000 = 0,0032688

Memory

Kilowatt hours = [Memory (GB) exceeding SPECPower database average] x [Memory coefficient] x usage amount (Hours) * powerUsageEffectiveness

 [Memory (GB) exceeding SPECPower database average] = 6.13 (I did the calculation using the family of the AWS instance: m5a)

[Memory coefficient] = 0.000392, // kWh / Gb

powerUsageEffectiveness = 1.135

usage amount: 24hs

6.13 * 0.000392 * 24 * 1.135 = 0.065

Networking

Kilowatt hours: gigaBytes * this.coefficient * powerUsageEffectiveness

gigaBytes = 53.8 Gb

this.coefficient =  0.001, // kWh / Gb

powerUsageEffectiveness = 1.135

53.8 * 0.001 * 1.135 = 0.061063

Total

(compute kwH + storage kwH + memory kwH + network kwH) * emissionFactor

 

Emission Factor (WECC): 0.000350861

( 0,0459 + 0,0032688 + 0.065 + 0.061063) * 0.000350861 = 0.00006 co2eq 

Embodied emission

Emission factor CO2eq = scopeThreeEmissions *

(usageTimePeriod / this.serverExpectedLifespan) *

(instancevCpu / largestInstancevCpu) =

For AWS m5a.large:

scopeThreeEmissions = 1.6103792

usageTimePeriod = 24hs (is it right?)

this.serverExpectedLifespan = 35040

instancevCpu: 2

largestInstancevCpu[m5a.24xlarge] = 96

1.6103792 * (24/35040) * (2/96) = 0.00002

Then, the final number should be Total CO2e 0.00002 + 0.00006 = 0.00008 CO2eq

Thank you very much for your collaboration! Stay in touch!

Juli

[Cloud Carbon Footprint]

Hi Juli,

These calculations look good to me. Great work! Please continue to share any information you find that could be relevant or useful for the CCF tool going forward.

Thanks,

Cam