Influencing factors #

Central question

• What are the influencing factors, and how do they affect the ecological footprint of AI?

The following section presents impact factors organized into five primary categories: use case, model, data, hardware, and tools. In the context of AI systems, these impact factors refer to specific components or areas within the system that can significantly contribute to its environmental impact. These factors may include, but are not limited to, the energy consumption of data centers, the carbon footprint of hardware production, the resource utilization during operation, and the ecological effects of disposing electronic waste.

Use Case #

AI applications are designed for specific situations in which the AI is employed to achieve particular user goals, which are referred to as use cases. In this category, we have organized the impact factors that are related to the problem solution.

There are a multitude of stakeholder groups involved in the life cycle of an AI application. The following is a brief list of stakeholders and their potential influences on the environmental impact of AI solutions.

• Developers: This group refers to all technical stakeholders that have a direkt or indirekt influence on the AI implementation (e.g. DevOps, etc.). They can exert effort into the efficiency of the model, implementation, used frameworks, etc.
• Executive officers: They can influence the overall goal of the AI application may attach value to the ecological footprint.
• Users: Users, purchasers, customers, etc. may opt for AI systems with a lower environmental footprint over applications that might perform slightly but have a worse footprint. However, this requires the user group to be aware of the environmental impact of the AI system.
• Politics: Political decision makers have the capacity to influence the manner in which AI systems are developed throughout their entire life cycle. By furnishing a comprehensive compilation of the environmental impacts associated with AI systems, they are able to incorporate Green AI into this process and to establish standards and regulations for AI that is more ecologically sustainable.

The user experience (UX) of an AI application can significantly influence its environmental impact. For example, if the application’s interface is unclear, users may inadvertently execute resource-intensive inference processes more often due to misunderstandings. To address this, the design could facilitate user awareness of the environmental consequences of their actions. One potential solution is to provide users with the option to select between models of varying strengths and ecological footprints. Furthermore, enhancing transparency about the expected quality from each model can empower users to make informed decisions that balance performance with sustainability.

The rebound effect is a phenomenon where the gains from more efficient AI applications are offset by changes in behavior that result in an increase in overall consumption. There are a number of reasons for this. One potential explanation for this phenomenon is that increased efficiency may lead to increased use. For instance, a chatbot powered by a LLM may exhibit a lengthy response time, which may dissuade users from utilizing it due to their impatience. Improving the efficiency of the LLM may also result in a faster response time, which could lead to an increase in the use of the chatbot. Additionally, there are indirect effects. For instance, if AI results in significant cost savings in one area, a company may choose to invest the saved resources in other areas that also consume energy or resources, such as expanding their data centers or increasing manufacturing.

In certain scenarios, machine learning (ML) methods demonstrate superior performance due to their ability to address complex and dynamic problems. Nevertheless, it is not always the case that ML represents the most efficient solution. For example, a simple stochastic or deterministic method may be capable of solving some problems with less computational overhead, which could result in lower energy consumption. This is particularly the case in situations where decisions can reliably be made using clear, deterministic rules. For example, the assessment of eligibility for a loan based on specific financial thresholds may be more efficiently conducted through the application of rule-based systems. These systems are not only more straightforward but also offer greater transparency compared to ML models, which can be unnecessarily complex and less interpretable for such tasks. In light of these considerations, it becomes evident that the pivotal question in this context is “Does the problem in question even necessitate a machine learning solution?”

Related Work:

• [Alzoubi2024] provide a list of Green AI initiatives.
• [Cruz2024] consider the business case of the Green AI application.
• [You2023] benchmark various open source LLMs in terms of how much time and energy they consume for inference.

Model #

The fundamental element of each AI application is the AI model, which encompasses the algorithm, architecture, and training and inference methodologies.

The first impact factor is the size of the model. In general, larger AI models possess a greater number of parameters. These are the elements within the model that are adjusted during the training process. A greater number of parameters implies that the model must process a greater quantity of data in order to perform tasks such as training, fine-tuning, and inference. This necessitates the utilisation of greater computational resources, which, in turn, increases the energy required to execute these processes. Furthermore, larger models typically require longer training periods. Training involves running large datasets through the model multiple times with the objective of optimising the parameters in order to achieve accurate predictions. This prolonged training period consequently leads to a higher energy consumption, as the high-performance computing resources are engaged for a longer duration. During the inference phase, the model size continues to exert a significant influence on its energy consumption. While the model size contributes to increased energy consumption during inference, strategies such as utilizing sparsely activated models, like Mixture of Experts, can mitigate some of the computational demands. Although these methods help to reduce the operational footprint, the overall size of the model—and thus the environmental impact—remains substantial.

During the training phase, a number of decisions can impact the environmental footprint of the AI model. Hyperparameter tuning is designed to identify optimal parameters that achieve the highest accuracy, but this can increase the footprint. For example, choosing a sigmoid activation function over a simple binary function can lead to higher energy consumption due to the sigmoid’s greater computational intensity. Similarly, selecting the optimal neural architecture involves a trade-off: more complex architectures may yield superior results but demand more energy, whereas simpler architectures require less computing power but might produce slightly inferior outcomes. The architecture selection process itself significantly influences the AI’s footprint, as it often necessitates running training processes with various architectures to predict their performance, thereby increasing energy consumption if these runs are unnecessary. Training a model from scratch can be very energy-intensive. In contrast, methods that leverage pre-trained models, such as transfer learning, usually require less time and resources. This approach often improves performance, especially when data is scarce. For instance, transfer learning involves reusing a model developed for one task as the starting point for another task, reducing energy demands through fewer training iterations. However, the resulting model used for inference might be larger than a model specialized for a specific purpose only. In the event that the model is utilized for inference on a frequent basis, a model that has been trained from scratch may ultimately result in a smaller overall footprint than a model that has been trained using transfer learning, despite the higher initial energy footprint associated with the former. There are numerous methods for utilising pre-trained models, including transfer learning (e.g., fine-tuning), the use of pre-trained embeddings, feature extraction, ensemble methods, and knowledge distillation, among others. Another method for reducing the number of training executions is through the use of multipurpose models, such as large language models (LLMs). The training of these models is costly, yet they are capable of handling a multitude of tasks with a single model. Nevertheless, the initial training and inference processes of these models may require a significant amount of energy, particularly in comparison to single-purpose models.

Following the training phase, the inference process assumes a pivotal role in the life cycle of an AI model, particularly in terms of energy consumption. Although the model is no longer engaged in the training process, each time it is applied to new data in order to make predictions or analyses, it consumes energy. The energy requirement for inference can be influenced by the model’s size. Larger models, with extensive parameters, tend to require more computational power to process data inputs, leading to greater energy use. Nevertheless, the deployment of techniques such as quantization, which reduces the precision of the model’s parameters, and model pruning, which removes redundant parameters, can effectively decrease the energy demands during inference. These strategies not only streamline the model but also maintain performance efficiency, thus mitigating the environmental impact of operating large AI models on an ongoing basis.

The initial energy consumption of training is significantly greater than that of a single inference. However, inference typically occurs at a much higher frequency than training. In order to identify the greatest potential for energy savings, it is necessary to consider a number of factors, including the use case (e.g., the frequency of model execution), the hardware (e.g., whether the inference is executed on energy-efficient specialized hardware), and other relevant factors. A detailed examination of the complexity of these considerations is presented the next section.

• Energy demand LLMs https://ml.energy/leaderboard

Data #

In AI systems, the data utilized plays a significant role, influencing both the system’s effectiveness and its environmental footprint. The primary factors within the “Data” category include data size, data selection, and data collection. Each of these factors has an impact on energy consumption.

Data size involves considerations not only about the volume (number of records) but also the type of data (e.g., float32 vs float16). The processing of larger datasets or those containing higher-precision data types, such as float32, necessitates the allocation of greater computational resources. This results in an increase in energy consumption during the training of the model, as more extensive or higher-precision datasets necessitate more memory and processing cycles. Conversely, the utilisation of lower precision data types, such as float16, can result in a reduction in the memory footprint and computational load, thereby lowering energy consumption. However, this may impact the model’s accuracy and performance.

Selecting data can influence energy usage through the quality of the data chosen for training. The selection of representative and well-curated datasets can reduce the necessity for retraining and additional preprocessing, which can significantly consume energy. Selecting an appropriate dataset can reduce the number of iterations required to refine the model, thereby conserving energy. In contrast, the use of datasets of inadequate quality or lacking relevance to the intended problem-solving task can result in excessive computational overhead due to the processing of irrelevant data.

The process of data collection presents certain complexities, with key decisions to be made between the use of real-world data and synthetic data, as well as considerations regarding the frequency and level of detail to be incorporated into the data collection. Real-world data offers significant utility due to its genuine nature. However, the collection of such data can be energy-intensive, particularly when it necessitates the deployment of numerous sensors, frequent updates, and detailed data. Synthetic data, generated through simulations, can be produced with less energy expenditure and yet still provide valuable insights, particularly when the use of real-world data is either prohibitively expensive or not feasible. Nevertheless, the generation of high-quality synthetic data can also necessitate a considerable expenditure of computing resources.

Each aspect of data management—from the size and type of the data, through how it’s selected, to the way it’s collected—has a profound impact on the performance of the AI systems, as well as on their energy consumption and, consequently, their environmental footprint.

Hardware #

In AI applications, the choice and management of hardware assume significant importance with respect to both performance and environmental impact. This discussion explores the various hardware-related factors that influence energy consumption and the overall environmental footprint of AI systems.

The primary factor influencing the energy demand of an AI system is the energy efficiency of the used hardware, which can exhibit considerable variability. There are chips, such as CPUs, that are capable of covering a broad application space but have high energy consumption. At the opposite end of the spectrum, ASICs (application-specific integrated circuits) are designed for specific applications and are highly energy-efficient (e.g. Etched’s transformer chip, called Sohu). Within this range, there are other types of processing units designed for AI use cases, such as TPUs (Tensor Processing Units), NPUs (Neuromorphic Processing Units), GPUs (Graphics Processing Units), and more. Originally designed for computer graphics, GPUs are also highly applicable to AI due to the similarities in the operations required for both fields.

In addition to their energy efficiency, the environmental impact of the hardware is also shaped by their embodied emissions. These emissions refer to the totality of greenhouse gases released throughout the life cycle of the hardware that result from non-operational processes, such as extraction of raw materials, manufacture, assembly, transport, and disposal. The assessment of embodied emissions in computer hardware is challenging due to a number of factors. Firstly, computer hardware is composed of numerous components sourced globally through complex supply chains, making it challenging to accurately track and calculate total emissions. Additionally, there is a considerable disparity in the manner in which emissions data is reported—if it is reported at all—due to the absence of standardized data and the proprietary concerns of manufacturers and suppliers. Furthermore, the diverse manufacturing processes contribute to differences in the carbon footprints of similar products. This is due to the fact that manufacturers may use different production techniques, materials, and energy sources.

Another key factor in the hardware category is the compute location, which encompasses several aspects. These include the geographical location, the specific server or computer used, and concepts such as cloud computing and edge devices. Due to the varying energy mix (proportion of renewable and fossil energy sources) across different regions, the same hardware and software could emit considerably more CO₂ in one location than in another. Consequently, selecting a geographical location with a better energy mix could significantly reduce CO₂ emissions. Furthermore, even within the same region, prioritizing more energy-efficient hardware could result in a reduction in emissions. This is particularly the case when the execution does not require immediate completion and can instead await the availability of more efficient hardware. Finally, there is a significant debate surrounding the question whether or not it is more energy efficient to run AI algorithms in the cloud or on edge devices. The efficiency of AI algorithm execution is highly dependant on the specific circumstances under consideration. These include not only the hardware’s power usage effectiveness (PUE), which may be superior in the cloud, but also the additional data transfer to the cloud. This data transfer is unnecessary for edge devices, which are capable of processing data at the point where it is generated. A more detailed examination of this topic is presented in the next section.

Related work: [Wu2022] offer a comprehensive exploration of the environmental impact of AI growth, considering data, algorithms, and system hardware. They evaluate the carbon footprint of AI computing, highlighting the potential for hardware-software design and optimization to mitigate environmental impact. They also state that a majority of GPUs in their assessments were run only at 30-50%, causing inefficient idle time and possibly higher efficiency in managed cloud environments.

Another tool that can enhance the efficiency of AI applications is in-memory computing. It enables faster access and processing of data by storing it directly in the RAM instead of slower disk-based storage. This can result in a reduction in the time and energy required for data-intensive operations such as training large AI models, which in turn leads to a decrease in overall power consumption.

In addition, improper hardware settings can affect the power consumption of AI algorithms. For instance, reducing the power limit may result in an increased overall runtime, yet a reduction in the total energy consumption.

In the context of cloud computing, the operation of hardware extends beyond the mere consumption of electrical energy. It also involves the use of additional resources. For instance, water is commonly employed in cooling systems. This dependence on water cooling can give rise to considerable environmental concerns, particularly in areas where fresh water is in short supply or if the heated water from cooling systems is fed back into rivers, raising their temperature and impacting biodiversity.

Tools #

In the domain of AI development and deployment, the selection of appropriate tools is crucial for the optimization of both performance and environmental impact. This section examines the influence of various tools on the sustainability of AI applications.

AI developers have access to a plethora of frameworks that can influence the efficiency and environmental impact of their projects. Popular frameworks such as TensorFlow, PyTorch, and Keras offer different capabilities in terms of ease of use, flexibility, and performance. Furthermore, each framework exhibits varying degrees of efficacy in resource management, which can have a pronounced impact on energy consumption during model training and inference.

The implementation of practices that enhance energy and resource awareness can result in a notable reduction in the environmental impact of AI systems. For example, scheduling training sessions during off-peak hours, when there is less demand on the power grid, or in regions where CO₂ emissions from energy generation are lower, can result in the use of more renewable energy sources and lower carbon emissions (sources, such as electricitymaps.com or SMARD (see also [Steinberg2020)] can be used to automatically schedule or shift code execution to times/locations with lower CO₂ emissions per kWh). This approach not only optimizes energy usage but also potentially reduces costs.

The methodologies used during the development process, such as Scrum and Agile, can also influence the environmental impact of AI applications. These methodologies promote iterative development, continuous feedback, and adaptability. By facilitating more effective project management and expedited identification of issues, these approaches can minimize wasted effort and resources, thereby reducing the environmental impact of the development process.

MLOps, or Machine Learning Operations, represents a pivotal framework for the management of the life cycle of machine learning models. However, if it is not implemented carefully, it can exacerbate environmental impacts. The integration of machine learning systems with continuous integration and continuous delivery (CI/CD) pipelines enables the automation and streamlining of model development, deployment, and maintenance through the implementation of MLOps. However, in the absence of effective monitoring and management practices, these operations can result in the inefficient use of computing resources and increased energy consumption. The automation of repetitive tasks, while enhancing model deployment precision, can also result in excessive resource use if not properly calibrated. Furthermore, MLOps encourages the adoption of reproducible workflows, which can result in predictable energy usage patterns. However, if scalability and resource demands are not accurately anticipated, these workflows can become environmentally costly. Inappropriate implementation of MLOps can result in an increased environmental footprint due to the mismanagement of resource allocation and usage across the machine learning model lifecycle. In particular, the occurrence of data and concept drift may result in the need for unnecessary retraining sessions. The scheduling of frequent retraining, such as nightly updates despite minimal drift and acceptable model accuracy, can result in significant and unnecessary energy consumption.

References #

• [Alzoubi2024]

  Alzoubi, Y. I., & Mishra, A. (2024). Green artificial intelligence initiatives: Potentials and challenges. In Journal of Cleaner Production (Vol. 468, p. 143090). Elsevier BV.

• [Cruz2024]

  Cruz, L., Gutierrez, X. F., & Martínez-Fernández, S. (2024). Innovating for Tomorrow: The Convergence of SE and Green AI (Version 1). arXiv.

• [Steinberg2020]

  Steinberg, D., Murach, J., Guldner, A. & Gollmer, K.-U. (2020). Online energy forecasts for the Internet of Things. In New Perspectives in Environmental Information Systems: Transport, Sensors, Recycling. 34th EnviroInfo (pp. 165–174)

  Erneuerbare Energie und Sektorkopplung als Schlüssel zur Energiewende. https://www.umwelt-campus.de/iot-werkstatt/tutorials/klimaschutz-iot-stromboerse-und-co2, https://datenpuls.umwelt-campus.de/ (German only).

• [Wu2022]

  Wu, C.-J., Raghavendra, R., Gupta, U., Acun, B., Ardalani, N., Maeng, K., Chang, G., Behram, F. A., Huang, J., Bai, C., Gschwind, M., Gupta, A., Ott, M., Melnikov, A., Candido, S., Brooks, D., Chauhan, G., Lee, B., Lee, H.-H. S., … Hazelwood, K. (2022). Sustainable AI: Environmental Implications, Challenges and Opportunities(Version 2). arXiv.

• [You2023]

  You, J., Chung, J.-W., & Chowdhury, M. (2023). Zeus: Understanding and Optimizing GPU Energy Consumption of DNN Training. In 20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 23) (pp. 119–139). USENIX Association. https://www.usenix.org/conference/nsdi23/presentation/you

  Chung, J.-W., Liu, J., Wu, Z., Xia, Y. & Chowdhury, M. (2023). ML.ENERGY Leaderboard. https://ml.energy/leaderboard

  You, J., Chung, J.-W., & Chowdhury, M. (2022). Zeus: Understanding and Optimizing GPU Energy Consumption of DNN Training (Version 2). arXiv.