Using CCF Methodology to calculate footprints by hand

154 views
Skip to first unread message
Assigned to cameron...@thoughtworks.com by dan.lewi...@thoughtworks.com

Elie

unread,
Nov 11, 2021, 4:39:42 PM11/11/21
to Cloud Carbon Footprint

Hello, 

I am considering using the ccf calculator for professional use. But I was curious to estimate my current company CO2 emissions on private and on-prem DCs. So, I tried to manually calculate our emissions with your methodology. But the results I got are well under what I expected. 

You will find below the calculation I applied to my data. I believe I made a mistake somewhere, but I can’t find it. 

For simplicity sake, I will not detail the different coefficients, but I applied different coefficients depending on several factors such as DC's size, location and other parameters such as PUE. 

Calculation:

Total CPU Calculation in Wh:

= Total number of CPU * Number of Hours in Year * Average CPU Watts

With:
Average CPU Utilization = 0,5

Number of Hours in Year = 8760

Average CPU Watts = 2,27

Total Storage Calculation in Wh:

= Total SSD in GB * (SDD energy estimation (Wh/TB)/1000) + Total HDD in GB * (HDD energy estimation (Wh/TB)/1000)*Number of Hours in Year

With: 

SSD energy estimation (Wh/TB) = 1.2 

Number of Hours in Year = 8760

Total Memory Calculation in Wh:

= Total RAM in GB * Memory Watts per GB * Number of Hours in Year

With: 

Memory Watts per GB = 0.392

Number of Hours in Year = 8760

Total Wh Calculation: 

= Total CPU Calculation in Wh + Total Storage Calculation in Wh + Total Memory Calculation in Wh

Total Carbon Emission Calculation:

=(Total Wh/1000) * Grid emission factor (tCO2/kwh)


My calculations miss some elements and other elements (as RAM) will be overvalued, but the result I get from this calculation is very low compared to what I expected. 

Do you think this calculation is correct or did I miss something? 

Sincerely, 

Elie


Cloud Carbon Footprint

unread,
Nov 29, 2021, 2:04:44 PM11/29/21
to Cloud Carbon Footprint

Hi Elie,


Thanks for reaching out!

 

We have not completed work to calculate on-prem emissions just yet so we are very excited to get this going and have been looking to work with a contributor to configure a good methodology!

Would you be interested in setting up a call to discuss in more detail? In the meantime, I have provided some notes:

 

Compute:

 Our Methodology:

 Kilowatt hours =  ((minWatts + (averageCPUUtilization / 100) * (maxWatts - minWatts)) *

   virtualCPUHours *

   powerUsageEffectiveness) /

1000 // convert to kWh

 ((minWatts + (averageCPUUtilization / 100) * (maxWatts - minWatts):  I am thinking your value of 2.27 is too low. With our methodology, we are working with virtual CPU's so we are dividing the min/max watts by the number of threads. Since I believe you are working with physical CPU's rather than virtual, you should not need to divide by the threads. Instead, you should use the average across the entire SPECPower DB, or if you know what microarchitecture you are using, take the average of that subset. You can even go even more granular if you know exactly which processors are used.

Here you can reference the SPECPower Database: https://github.com/cloud-carbon-footprint/cloud-carbon-coefficients/blob/main/data/SPECpower-full-results.csv

I'm thinking your min/max watts should ideally be at least 100x higher so hopefully this makes sense.

virtualCPUHours: in your case i'm thinking this may be 24 hours (per day) * number of physical server chips which could be determined from your underlying micro-architecture or looked up in the SpecPower DB

PUE: this would need to be determined probably somewhere in the range of 1.2 - 1.6


Storage:

Our Methodology:

 Kilowatt hours = (terabyteHours *

  this.coefficient *

  powerUsageEffectiveness /

1000 // convert to kWh

 

I'm thinking here you may just need to convert your total SSD/HDD in GB to TB Hours based on hours per day which would be:  (usageAmount / 1000) * 24

Then you would use the correct coefficient based on SDD: 1.2 or HDD: .65

as well as the determined PUE

 

Memory:

 Our Methodology:

 Kilowatt hours = Memory usage (GB-Hours) x Memory coefficient

 

I think you have this correct at a high level, I would again just use hours in a day 

 

Hope this helps!

 

Thanks,

Cam

Elie

unread,
Dec 2, 2021, 10:40:59 AM12/2/21
to Cloud Carbon Footprint

Hi CCF,

Thank you for your answer!

On compute:

You are right, I recalculated my results with the ratio of min & max watts/chips, and the results look way more coherent.

For service providers, Average CPU Wh = 65

For local DC, Average CPU Wh = 53

Unfortunately, I don’t have access to the type of micro-architecture.

For memory:

I have also adjusted my calculations to avoid overestimating the RAM energy consumption. Using the following Excel formula:

IF(Sum of RAM/Sum of Chips> Average Ratio RAM/Chips; Sum of RAM-(Sum of Chips* Average Ratio RAM/Chips);0)

Average Ratio RAM/Chips = 51 (source: SPEC DB)

 

Concerning meeting to discuss a methodology, I would be happy to discuss this issue in detail and help develop a method for on-prem clouds. But I first have to get approval from my hierarchy. Hence, I will get back to you soon with an answer.

For the time being, I would prefer to keep our discussion private, of course, you can republish your insights and mines (anonymously). 

Sincerely yours,

Elie 

Julian Len

unread,
Dec 2, 2021, 4:33:22 PM12/2/21
to Cloud Carbon Footprint
Hi!

I'm also interested in calculating consumption by hand. 

On Compute, watching at the code there is a "replication factor". Why did not you get it into account?

Thanks!

Cloud Carbon Footprint

unread,
Dec 2, 2021, 4:51:06 PM12/2/21
to Cloud Carbon Footprint
Hello!

We are very open to any suggestions as we look to implement an approach for calculating consumption by hand.

In Elie's case, I made the assumption that there are no back up servers which is where we would consider a replication factor. In the cloud, there is the concept of having redundancy with multiple availability zones to make your cloud process fail-proof. Depending on the storage service, we have varying replication factors. You can view how we determine those here:
https://www.cloudcarbonfootprint.org/docs/methodology#storage-services-replication-factors

Thanks,
Cam

Elie

unread,
Dec 3, 2021, 5:27:36 AM12/3/21
to Cloud Carbon Footprint
Hello Cam, Jul,

Indeed, I took into account the replication factor directly in the amount of storage I provisioned. 

Sincerely, 
Elie

Cloud Carbon Footprint

unread,
Dec 6, 2021, 2:22:28 PM12/6/21
to Cloud Carbon Footprint
Hi Elie, Jul,

Our team is going to plan a concerted effort towards calculating on-premise footprints. Would you both be interested to setting up a call to discuss ways to collaborate in more detail? We would be happy to meet this week or next!

Thanks,
Cam

Elie

unread,
Dec 7, 2021, 11:42:33 AM12/7/21
to Cloud Carbon Footprint
Hi Cam,

Let me come back to you by tomorrow! 

Sincerely, 
Elie

--
We're excited to hear from you and hope you're enjoying Cloud Carbon Footprint.
Please fill out our feedback form: https://forms.gle/nFzkRioryy4R1DGB6
Add your name to ADOPTERS.md: https://github.com/cloud-carbon-footprint/cloud-carbon-footprint/blob/trunk/ADOPTERS.md
Give us a star on the github if you're enjoying the tool!
---
You received this message because you are subscribed to a topic in the Google Groups "Cloud Carbon Footprint" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cloud-carbon-footprint/rqvHCFjw1Fk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to cloud-carbon-foot...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-carbon-footprint/898863ae-30a5-4904-936b-b3f9d76520b4n%40googlegroups.com.

Elie

unread,
Dec 8, 2021, 7:07:13 AM12/8/21
to Cloud Carbon Footprint
Hi Cam, 

As promised, I am getting back to you. And I am pleased to tell you that our answer is positive. 

Also, I am currently working with Accenture on the subject and would like to invite them as well if you agree. 

We can discuss the date of the meeting by email, with: 
- my professional email: elie.de...@loreal.com 
- and the email of the Accenture consultant: martin...@accenture.com

Sincerely yours, 
Elie de Perthuis

Elie

unread,
Dec 14, 2021, 8:54:56 AM12/14/21
to Cloud Carbon Footprint
Hello Cam, 

I get back to you about the meeting on the method to assess the carbon footprint of on-prem DC. 

Indeed, here in France most of us are soon (17/12) in Christmas holiday. Hence, it would be preferable to meet before the end of the week. If not possible, I personally am available until 31/12 

Sincerely yours, 
Elie 

PS: We would be glad to share information with you, I strongly believe it would be mutually beneficial! 

Message has been deleted

Julian Len

unread,
Jan 4, 2022, 2:39:52 PMJan 4
to Cloud Carbon Footprint
Hey everyone!

Hope you are doing well.

Why aren't you using 'power usage effectiveness' in the memory consumption calculation?

Sincerely yours,

Juli

Cloud Carbon Footprint

unread,
Jan 4, 2022, 2:52:08 PMJan 4
to Cloud Carbon Footprint
Hi Juli,

We are factoring in PUE for memory estimations. See here:
https://github.com/cloud-carbon-footprint/cloud-carbon-footprint/blob/50bd22104dcb42a20bc4314ca3d7e6403aff4a95/packages/core/src/memory/MemoryEstimator.ts#L55

I had just left it out above earlier in the thread.

Thanks,

Julian Len

unread,
Jan 4, 2022, 3:00:19 PMJan 4
to Cloud Carbon Footprint
Thanks for the quick response!

I meant when the formula to calculate the CCF for the memory was presented, it was

Memory:

 Our Methodology:

 Kilowatt hours = Memory usage (GB-Hours) x Memory coefficient

Which I think the PUE was forgotten.

Finally today I've arrived at a number for each category, I'll post it above later

Thank you!

Juli

Message has been deleted

Julian Len

unread,
Jan 5, 2022, 5:02:11 PMJan 5
to Cloud Carbon Footprint
Hi everyone!

My apologies for the number of messages, I hope to be closer!

Finally, I arrived at a number. This is the procedure I followed:

The company I worked at provided me the next information regarding the AWS instance:
Instance family and type: EC2 m5a.large
Processor: AMD_EPYC_1ST_GEN
Storage used: 230 GB SSD (not the total allocated)
Region: us-west-1-North California.

In all the cases, I assumed a replication factor of 1.


Compute

Average Watts = ((minWatts + (averageCPUUtilization / 100) * (maxWatts - minWatts))
Compute Watt-Hours = Average Watts * vCPU Hours
Kilowatt hours [kWh] =   Compute Watt-Hours *
   powerUsageEffectiveness) / 1000 // convert to kWh

minWatts[AMD_EPYC_1ST_GEN] = 0.82
maxWatts[AMD_EPYC_1ST_GEN]: 2.55
averageCPUUtilization: 50,
powerUsageEffectiveness: 1.135
Regarding vCPU hours I used 24hs. Is it correct? Since it is an instance working 24 hours a day.

Storage
We are using an SSD

Kilowatt hours = terabyteHours *  this.coefficient *  powerUsageEffectiveness /1000 // convert to kWh

terabyteHours = Our instance has occupied 230GB, do I have to calculate totalOccupied*24h?
this.coefficient = 1.2 Watt-hours (SSD)
powerUsageEffectiveness = 1.135

Memory

Kilowatt hours = [Memory (GB) exceeding SPECPower database average] x [Memory coefficient] x usage amount (Hours) * powerUsageEffectiveness

 [Memory (GB) exceeding SPECPower database average] = 6.13 (I did the calculation using the family of the AWS instance: m5a)
[Memory coefficient] = 0.000392, // kWh / Gb
powerUsageEffectiveness = 1.135
usage amount: Regarding usage amount, I am a bit lost here. Do I need to know how much memory has my instance? Or just use "24hours"?

Networking

Kilowatt hours: gigaBytes * this.coefficient * powerUsageEffectiveness

gigaBytes = The only information I have is NetworkIn and NetworkOut. 
Network In is 108GB while NetworkOut is 53.8GB. What should I put?  AFAIK, this is the total usage of the instance. So, should I use the total usage of the instance but in a day? 
this.coefficient =  0.001, // kWh / Gb
powerUsageEffectiveness = 1.135

When the "emission factor" is calculated, what is the resulting unit? 

Do you think it is correct? I followed your answer here and the codebase at the same time. I wrote the questions in red so it is easier to see.


Thank you very much!

Greetings.

Juli

James Hall

unread,
Jan 6, 2022, 12:01:08 PM (13 days ago) Jan 6
to Cloud Carbon Footprint
I've been trying something similar and this thread has helped a lot but when I work out the computation carbon it comes out very low considering in the demo graph it says that 54 KWhs of one AWS account using EC2 will release 3 metric tonnes of CO2 into the atmosphere


Min watts        107.414286
Max watts        331.985714
averageCPUUtilization        50%
CPU hours        200
PUE        1.2
Average watts        219.7
Compute Watt-Hours        43940
Kilowatt hours [kWh]         52.728
GEF of england (tonnes)        0.000235
CO2e (tonnes)        0.01239108

Any assistance in where I'm going wrong would be helpful. I've seen in this thread that the chips need to be taken into account but it doesn't appear to mention it in any equations in the methodology page

Oran Maliphant

unread,
Jan 13, 2022, 6:19:27 AM (7 days ago) Jan 13
to Cloud Carbon Footprint
Just wondered if there was any methodology that incorporates GPU power usage as well as CPU/Memory/Disk ?
our org has heavy GPU users as well as 3rd party hardware that are developed for specific number crunching tasks.
has anyone solved for this yet?

Reply all
Reply to author
Forward
0 new messages