Help on Impact Framework for Student Project

[Martha Calder-Jones]

Hi there, 

I'm a CS Master's student at UCL about to start my final project, which involves using the IF on a virtual machine with telemetry data (GPU, CPU, RAM) to gain insight into how sustainable the system(s in a workspace) is (are) running, and answer other related Qs.

I'm trying to understand the detail in which the IF models can provide insight, and in what format this insight comes out in. I haven't yet got my hands on any dataset (will do soon!), but am keen to understand this part ASAP.

I'll be looking to have users be able to get answers to soft language questions like:

What is the unit that is more intensively using GPU?

Give me a summary of the compute usage within the units’ pool over the last day/week/month?

My gut is leaning towards doing something which is a combination of IF and somehow using the Langchain framework. At this point, I have no idea how that would look. 

Also, if anyone is able to share the extent to which the parameters in the models can be amended, and the pros and cons of doing so, I'd be really grateful (I've seen this page, but would love to get a better understanding of what this all means!).

The project is in its first days and I have LOTS to learn, so I'd love to hear anyone's thoughts on this!

Thanks so much,

Martha 

[Asim Hussain]

Hi Martha, 

Welcome! 

GPU is an interesting story, I don't think we have a consensus view on this yet but I'd recommend you take a look at some of the hackathon entries this year that focussed on AI measurement:

• Create a manifest file for training an LLM hack#63 (Winner!)
• Create a model plugin to calculate the energy and carbon consumed during AI model training hack#76
• Impact Framework for LLMs: Injecting transparency into the rise of GenAI hack#130
• Measuring & Visualising the environmental impact of LLMs using the Impact Framework hack#135
• Plugin Search and Composition Using NLP hack#145

Most solutions, if I remember correctly, used direct energy measurements using an nvidia tool and then used an IF pipeline to process the energy into other impacts.

There is also an issue being discussed in the real-time-cloud project here, that's the GSF team that works to get consensus on the right way to compute different metrics:

https://github.com/Green-Software-Foundation/real-time-cloud/issues/37

If you've got questions about IF it always helps to drop in a manifest file or some snippets to help people understand the nature of the question :)

Cheers,

Asim

--
You received this message because you are subscribed to the Google Groups "[public] if-community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to if-community...@greensoftware.foundation.
To view this discussion on the web visit https://groups.google.com/a/greensoftware.foundation/d/msgid/if-community/0c585298-43a4-4bc0-9c52-d2d086467fean%40greensoftware.foundation.

--

Asim Hussain

Executive Director

Green Software Foundation

LI | TW 

[Martha Calder-Jones]

Thank you Asim for your reply - I really appreciate it! 

I've put together this first attempt at a manifest file, and I'm going to want the output in csv format.

I see there are a couple of ways to do this, however, I'm getting a 'key not found for columns' error when I try the simple addition of csv to outputs (a file generated with just PATH in it), and then a 'required' error for the yaml-to-csv method...  perhaps I've structured something incorrectly in the file? 

For calculating SCI, we need time-reserved as input... I don't know what this means in the context of a VM being used in general, does that make sense? I might be misunderstanding something here!

Any pointers, advice, feedback from anyone would be very appreciated! :)

Thanks a lot,

Martha 

name: sci-calculation

description: Calculate operational carbon from CPU utilization using the Teads curve and then get operational and embodied carbon too

initialize:

  # outputs:

  #   - csv

  plugins:

    group-by:

      path: 'builtin'

      method: GroupBy

    # yaml-to-csv:

    #   method: Shell

    #   path: "@grnsft/if-plugins"

    interpolate:

      method: Interpolation

      path: builtin

      global-config:

        method: linear

        x:

          - 0

          - 10

          - 50

          - 100

        y:

          - 0.12

          - 0.32

          - 0.75

          - 1.02

        input-parameter: cpu/utilization

        output-parameter: cpu-factor

    cpu-factor-to-wattage:

      method: Multiply

      path: builtin

      global-config:

        input-parameters:

          - cpu-factor

          - thermal-design-power

        output-parameter: cpu-wattage

    wattage-times-duration:

      method: Multiply

      path: builtin

      global-config:

        input-parameters:

          - cpu-wattage

          - duration

        output-parameter: cpu-wattage-times-duration

    wattage-to-energy-kwh:

      method: Divide

      path: builtin

      global-config:

        numerator: cpu-wattage-times-duration

        denominator: 216000

        output: cpu-energy-raw

    calculate-vcpu-ratio:

      method: Divide

      path: builtin

      global-config:

        numerator: vcpus-allocated

        denominator: vcpus-total

        output: vcpu-ratio

    correct-cpu-energy-for-vcpu-ratio:

      method: Divide

      path: builtin

      global-config:

        numerator: cpu-energy-raw

        denominator: vcpu-ratio

        output: cpu/energy

    sum-energy-components:

      path: builtin

      method: Sum

      global-config:

        input-parameters:

          - cpu/energy

          - network/energy

        output-parameter: energy

    embodied-carbon:

      path: builtin

      method: SciEmbodied

    operational-carbon:

      path: builtin

      method: Multiply

      global-config:

        input-parameters:

          - energy

          - grid/carbon-intensity

        output-parameter: carbon-operational

    sum-carbon:

      path: builtin

      method: Sum

      global-config:

        input-parameters:

          - carbon-operational

          - carbon-embodied

        output-parameter: carbon # total carbon in gCO2eq

    sci:

      path: builtin

      method: Sci

      global-config:

        functional-unit: machine

tree:

  children:

    child:

      pipeline:

        - group-by

        - interpolate

        - cpu-factor-to-wattage

        - wattage-times-duration

        - wattage-to-energy-kwh

        - calculate-vcpu-ratio

        - correct-cpu-energy-for-vcpu-ratio

        - sum-energy-components

        - embodied-carbon

        - operational-carbon

        - sum-carbon

        - sci

        # - yaml-to-csv

      config:

        group-by:

          group:

            - instance-type

        # yaml-to-csv:

        #   executable: python C:\Users\martha.calder-jones\OneDrive - University College London\UCL_comp_sci\Summer Project\HPSusSys\src\yaml-to-csv.py -c C:\Users\martha.calder-jones\OneDrive - University College London\UCL_comp_sci\Summer Project\HPSusSys\output.csv -j

      defaults:

        device/emissions-embodied: 1533.120 # gCO2eq (random for now) and is the sum of Life Cycle Assessment (LCA) emissions for the component

        time-reserved: 3600 # 1hr in seconds

        grid/carbon-intensity: 35 # this is the number for London (June 2024)

        device/expected-lifespan: 94608000 # 3 years in seconds == the length of time, in seconds, between a component's manufacture and its disposal

        resources-reserved: vcpus-allocated

        resources-total: vcpus-total

        machine: 1 # this is for 1 machine right now as have taken the average for machines rather than data for all machines

        duration: 21600

        # cloud/region: uk-west # don't think i need this as not using cloud metadata?

      inputs:

        - timestamp: 2024-07-14T00:00:00.000Z

          thermal-design-power: 125

          vcpus-total: 24 #Total VCPUs for Intel Core i9-12900K (8 P-cores and 8 E-cores with Hyper-Threading)

          vcpus-allocated: 1 # need to check

          cpu/utilization: 0.76

          network/energy: 0.000001 # this is random right now

          instance-type: c24

        - timestamp: 2024-07-14T00:00:00.000Z

          thermal-design-power: 165

          vcpus-total: 28

          vcpus-allocated: 1 # need to check

          cpu/utilization: 0.13

          network/energy: 0.000001 # this is random right now

          instance-type: c28

[Joseph Cook]

Hi Martha,

I just tried running your manifest locally and it executes correctly (with the csv export commented out as per your example). For exporting to CSV, you seem to be running some external script as a child process using the Shell plugin - it's hard to debug this as it's your own external program that we can't see. I'd recommend using the buiiltin CSV exporter instead unless you have some specific requirement that the builtin version doesn't meet.

There are instructions for how to export to CSV file here: https://if.greensoftware.foundation/users/how-to-export-to-csv

Cheers

Joseph

To view this discussion on the web visit https://groups.google.com/a/greensoftware.foundation/d/msgid/if-community/5c4798cd-9a65-4d83-8747-63ca68457135n%40greensoftware.foundation.

--

*****************

Joseph Cook

Product Owner: Impact Framework, 

Green Software Foundation

*****************