The ecological impact of AI: What's going on behind the scenes?

Every time we use - whether for generating or processing text, images or videos - we run energy-intensive processes in the background. These have an impact on the consumption of electricity and even water, which is used directly or indirectly for running and cooling computing devices.

In order to approach AI more responsibly, it is essential to understand what is going on behind the scenes. In doing so, we will not only gain an understanding that it is not an infallible source of truth, but also that it has an impact on the environment.

Prompt processing background

Our interaction with AI models has a significant resource dependency. But how exactly does our messaging with these models consume electricity, and what's more, water? The answer lies in a few key phrases and phases:

Learning the model

Before the model is ready to generate responses, it goes through a rigorous training process.

(LLM) learns through self-supervised learning. That is, the models are not trained on data with explicit labels, but learn to predict the next word based on the previous context. This approach allows models to efficiently use large unstructured texts.

The basic architecture of these models is transformers, which allow efficient sequence processing and the capture of long-term dependencies in the text. During training, the model traverses the text, predicts the following words, and adjusts its internal parameters based on errors. This process is performed on large-scale data including texts from the Internet, books, code, and other sources.

Two things are crucial for effective machine learning: good data and enough of it. With this, models learn to mimic human interaction by predicting what word might follow based on the previous context they maintain in their memory, known as the context window.

Environmental impact

However, the training is not only energy intensive for computing, but also water intensive. For example, training the GPT-3 model in Microsoft data centres consumed approximately in the form of steam emissions. This amount of water corresponds, for example, to the consumption needed to produce roughly.

This water consumption is also due to the need to keep servers in data centres at an optimal temperature. Cooling is often accomplished using water systems, where water absorbs the heat generated by the servers and then evaporates in cooling towers. This process is energy intensive, raising sustainability concerns in the context of the growing use of AI.