Skip to content
OpenGateLLM

Environmental footprint

OpenGateLLM tracks the environmental impact of AI model usage through the EcoLogits library, which provides a comprehensive view of the environmental footprint of generative AI models at inference.

The environmental footprint is deduced from the number of parameters of the model and the country where the model is hosted. For each model provider, you can define these parameters that impact the environmental footprint. Either through the Provider settings page in the Playground UI, or via the config.yml file. For more details on model configuration, see the model setup documentation.

Configuration

Section titled “Configuration”

There are three way to configure environmental impact tracking, by Playground, API or configuration file.

To define model parameters in the Playground, go to the Provider page. Complete the form fields when you create or edit a model provider including:

  • Total params of the model: Total number of parameters of the model in billions of parameters for environmental footprint computation.
  • Active params of the model: Active number of parameters of the model in billions of parameters for environmental footprint computation.
  • Hosting country of the model: Hosting country of the model in ISO 3166-1 alpha-3 code format (e.g., WOR for World, FRA for France, USA for United States).

Environmental impact metrics

Section titled “Environmental impact metrics”

For each call to a generative AI model, the API returns environmental impact metrics in the response, in the usage.carbon field. The metrics include minimum and maximum estimates to account for variability in model efficiency:

  • kWh: Energy consumption in kilowatt-hours (kWh), representing the final electricity consumption.
  • kgCO2eq: Global Warming Potential (GWP) in kilograms of CO2 equivalent (kgCO2eq), representing greenhouse gas emissions related to climate change.

Example response:

{
  "id": "chatcmpl-123",
  "object": "chat.completion",
  "created": 1677652288,
  "model": "my-language-model",
  "choices": [
    ...
  ],
  "usage": {
    "prompt_tokens": 10,
    "completion_tokens": 20,
    "total_tokens": 30,
    "cost": 0.000015,
    "carbon": {"kWh": 0.0001456, "kgCO2eq": 0.0000672 }
  }
}

After the request is processed, the carbon footprint of the request is store in usage table by the hooks decorator attached to each endpoint. See usage monitoring documentation for more information.

You can also see the carbon footprint of the request in the Usage page of the Playground.

Usage page

How it works

Section titled “How it works”

Environmental impact is calculated using the EcoLogits library through the compute_llm_impacts function. The computation takes into account:

  1. Model parameters:

    • Total number of parameters (model_total_params)
    • Active number of parameters (model_active_params, defaults to total params if not specified)

  2. Token usage: The number of output/completion tokens generated by the model

  3. Request latency: The time taken for the inference request (in seconds)

  4. Electricity mix: The carbon intensity of electricity based on the hosting zone (model_hosting_zone). The electricity mix includes:

    • ADPE (Abiotic Depletion Potential - elements)
    • PE (Primary Energy)
    • GWP (Global Warming Potential)

For more information about methodology, see EcoLogits documentation.