Understanding the Environmental Impact of Generative AI Services

The rise of generative AI: A quick review.  Though the term generative AI was coined in 2014,¹⁵ since the end of 2022 it has gained notoriety, and thus great interest, well beyond AI research. The sudden and significant rise of this term, however, should not conceal the long research history behind the technology.⁹ Developing GenAI models requires collecting data and learning from it, which includes first selecting the best model structure and learning algorithm for the given task and then applying this algorithm to the model and the data. The first step hides an expensive development process, as GenAI models are usually composed of several already-developed models. The second step is called training. Once a model has reached the targeted quality level through training, it can be used on new data, referred to as the inference phase. Training can require hundreds of graphics processing units (GPUs) running in parallel. For example, Stable Diffusion was trained with 258 GPUs and 64 central processing units (CPUs). Training proceeds in steps, with each step processing a batch of data in parallel and typically hundreds of thousands of steps required. Powerful models already existed before 2022, but the availability of these models as an online service, such as ChatGPT, led to the GenAI’s popularity. The massive new use of these services and the IT infrastructures that support them, with their high demand for electricity¹² and critical equipment, raises the question of sustainability: What is the environmental impact of this new digital usage, and in what order of magnitude?

The environmental impact of AI: An uncompleted work.  Since 2019, studies^25,38 have begun to evaluate the electricity consumption of the machine learning (ML) training phase, as well as the associated greenhouse gas (GHG) emissions.^7,19 The focus on training can be partly explained by the fact that ML models were seen as research projects rather than mass-market consumer products. The most common method to measure or estimate electricity consumption during the training phase of GenAI models is thermal design power (TDP). A manufacturing constant representing the maximum power of a component, TDP is used to estimate energy usage by multiplying the TDP of computing components by the total training duration. While straightforward, this method is limited, as it accounts only for GPUs and neglects other server components such as CPUs, memory buses, switches, and fans. An alternative involves directly measuring electricity consumption during execution using power meters (PMs). External PMs are the most accurate but require direct hardware access, which is often impractical. Software-based power meters (software PMs), such as RAPL or NVML, provide component-specific consumption data and are more precise than TDP-based estimates, but are still incomplete.²⁴ Both PMs share a significant drawback: They require measurements during execution, making pre- or post-execution estimates impossible. The literature thus lacks a methodology that is accessible, reproducible, reliable, and accurate for GenAI, the popularization of which has broadened the scope of its direct impacts on the environment.

As a consequence of the increasing usage of these models, researchers have also studied the impact of the inferences made from them^11,12,29—even if, similar to the training phase, this often occurs solely through measuring electricity consumption. Due to the digital sector’s large embodied footprint,¹⁸ including the full life-cycle impact is essential. Yet studies that include the full life cycle of equipment are still rare,^27,28 and are limited to the carbon cost of training and inference. The digital sector also has a strong environmental impact that stretches beyond GHG emissions⁵ to, for example, the extraction of rare metals. In addition, GenAI relies on and operates through the existing digital ecosystem, on which it logically exerts pressure, requiring terminals, such as smartphones and computers, for its users, as well as networks, to be accessible from its datacenters. All of these resources are essential for the deployment of GenAI and thus are part of the AI sustainability issue. It is therefore necessary to extend the scope of study on GenAI to view it as an accessible service with multiple environmental impacts.

LCA as an emerging tool for sustainability of computing.  Life cycle assessment (LCA) is a multi-criteria evaluation method based on the ISO 14040 and 14044 standards. It aims to evaluate the potential environmental impacts of a product or activity, considering all of its life cycle phases: manufacturing, usage, and end of life. As shown in Figure 1, an LCA is composed of four interdependent phases. The goal is to, for a defined purpose and perimeter (step 1), account for all the sub-products and elementary flows needed for the study’s subject (step 2), and sum their environmental impacts given by life cycle inventory (LCI) data (step 3). Step 4 questions the potential conclusions drawn with regard to the initial goals of the study and the uncertainties around the hypotheses made during the previous phases. Although LCA has long been used in other sectors,²¹ only recently has it been applied in the context of digital services,³⁰ where its benefits have been recognized. It enables a more comprehensive assessment by taking into account the complete life cycle and the different impact categories, thus avoiding focusing solely on the carbon emissions of the use phase. And though its use of assumptions has been criticized, it nonetheless enables relevant estimates to be made in the context of digital technology, where the industry’s lack of transparency⁴ could hinder the study of its impact on the environment. LCA therefore has the ability to question the sustainability of IT products and services, since it questions other sectors of activity using the same standard.

Generative AI as a service.  To better assess the environmental impact of AI, we propose studying not only the impact of developing a model but also that of its deployment and use as a service. Extending the model of Berthelot et al.,² Figure 2 summarizes the structure of a GenAI service. The arrows represent the data flows. GenAI users access services via personal terminals, sending requests that travel through networks to a Web server, where specific computational components perform model inference. The results are then returned to the user through networks. While the model undergoes a dedicated training phase requiring computational resources and training data, this study does not account for the environmental costs of producing this training data, as the process remains too opaque for analysis. For instance, the Stable Diffusion model, freely available as a service since August 2022, allows users to generate images by submitting prompts on its main Web page. The modular design of GenAI services enables adaptation to specific user scenarios; for example, a service hosted and used on a personal machine would remove the “Networks” and “Web hosting” sections. Following the recommendations of the standard for ICT services, this study aims to assess the environmental impact of running the service over a full year.

Estimating the electricity consumption of training through training step replication.  We propose a new approach to estimate the electricity consumption of the training phase. The existing methodologies described earlier are either unsatisfactory or require replication of the training, which would be too expensive for us. Our approach consists of replicating a fraction of the training while monitoring the electricity consumption and estimating the total training based on those observations. Anthony et al.¹ proved that the electricity consumption of epochs is constant. We show that this characteristic can be used to estimate the total training electricity cost by replication, assuming sufficient information from the original training. In this section, we illustrate our approach to Stable Diffusion. Several versions of the model exist, created by successive training phases from v1-0 to v1-5. We executed experiments on nodes from the Sirius cluster (Table 1) of the large-scale experimental Grid’5000 platform.⁸ This cluster was selected because of its similarity with the resources used by developers for the training and inference of the Stable Diffusion model. We used Ubuntu 20.04 with a default Nvidia GPU driver and monitored power consumption with an Omegawatt meter (0.1W precision, 1Hz) and ALUMET³¹ (2Hz). All results were averaged from seven experiments. The code and data are publicly available.²³ We replicated the v1-1 Stable Diffusion training on Sirius, using the same parameters except that the learning rate was kept constant. Assuming that the energy consumed by each node is equivalent, we carried out the experiments on a single node and extrapolated results. We used the Pokemon BLIP captions dataset and trained a linear regression model (R² > 0.99) to predict energy consumption based on the number of training steps (N) for 256×256 and 512×512 image resolutions (Equations 1 and 2, respectively).

Energy (kWh) = 1 . 78 e – 03 × N + 1 . 64 e – 02 (2)

Table 2 presents the estimated energy consumed by the model versions that are pertinent to this work. The obtained values are close to existing studies on similar models.²⁹

LCA of digital services.  Using the LCA of this service, beyond noting its significant environmental impact (463 tons of CO₂ eq.), we can also draw some information on how this impact is distributed. First, Figure 3 shows that terminals and networks represent a significant share in the impact of a GenAI service: more than 85% of the ADP impact, more than 30% of the energy footprint, and 45% of the carbon footprint. It validates the need to take them into account, all the more so since the footprint of networks and terminals grows with the number of users, even if users do not use an online service (that is, ignoring the cost of the network). Second, the multi-impact vision provided by LCA shows us that while decarbonizing the electricity sources of datacenters reduces GWP impact, embodied carbon emissions and those produced on the user side remain significant. The energy footprint is also a concern. Reducing it would require important efficiency gains, but could induce a rebound effect;¹⁰ that is, any progress on the energy efficiency of GenAI could well lead to an increase in its total usage. Lastly, the issue of metal extraction poses a difficult problem. GenAI services are boosting demand for GPUs, critical resources whose footprint is still difficult to estimate. In our study, we are using underestimated values for the footprint of GPUs, based on a method dedicated to CPUs detailed in Rince.³³

Obsolescence of hardware and software.  One of the most direct levers for reducing both ADP’s footprint and embodied emissions remains to extend the life of the equipment. If we consider the trend of the embodied carbon emission in IT,¹⁸ GenAI, with its high electricity consumption (shown in Figure 4), could indicate a reversal of tendencies. However, nothing about this conclusion is obvious, and it depends on how we consider the separation between operational and embodied impacts. Figure 5 represents training impact next to embodied impact, reflecting its one-time, resource-intensive nature similar to equipment manufacturing. It underscores the importance of extending the life cycle of both hardware and software, as training is a production cost for creating a model, much like software development.³⁷ We can also question the characterization of the network footprint. Admittedly, most of its footprint comes from electricity consumption. However, as explained in Guennebaud et al.,¹⁷ the transfer of data for a service does not directly generate additional consumption. Especially for fixed networks, the infrastructure has a basic cost that provides for needs as long as they remain within its capacity. Modifying the volume of data transferred by the service may in fact have little effect on electricity consumption,³⁵ unless it is on a scale that would require the infrastructure to be upscaled¹⁷ or unless the majority of the network is mobile. These last two points are not intended to show that a reduction strategy based on reducing data and energy flows could be ineffective. Such a strategy would, however, run the risk of being insufficient or contributing to a rebound effect. An approach based on the efficient and frugal use of existing resources could lead to more sustainable gains. That is, it may be more interesting to optimize existing resources than to seek to create new, supposedly more efficient resources.

Balancing between accessibility and reliability of the electricity consumption estimate.  We question the methodology for estimating training-phase electricity consumption and its scalability. In Figure 6, we compare our PM-based estimate of the v1-1 training electricity consumption with TDP-based and software-PM-based methods. Contrary to expectations, the TDP-based estimate was higher than the PM-based estimate (by 6%). Figures 7 and 8 compare power consumption measured by power meters and TDP to better understand this result. Figure 7 shows that GPU power consumption during 10 steps of training on the Sirius cluster is significantly lower than the TDP, despite high utilization. A TDP-based estimate overestimates GPU power consumption. Figure 8 shows the power consumption of 150 training steps on the Sirius (left) and Gemini (right) clusters, whose specifications can be found in Table 1, indicating that TDP-based estimates are unreliable and depend on workload and hardware. Figure 8 also shows that the difference between the software PM and the external PM is quite significant (around 20%), which results in a 25% difference when scaling to hundreds of thousands of training steps shown in Figure 6. To conclude, the best solution to estimate the electricity consumption of the training phase of a GenAI model is to be able to replicate the training step while monitoring with an accurate power meter. Without access to an external PM, software-based PMs provide an accurate measure of the computing components, but with a significant difference from external PMs, which is exaggerated by the number of training steps. If it is not possible to replicate the training steps but information such as the duration of the execution is available, using the TDP can provide an estimate but without guaranteeing its accuracy.