Greening Networking: Toward a Net Zero Internet

According to a recent EU report on Energy Consumption in Data Centres and Broadband Communication Networks in the EU [1], EU data center energy consumption is slowly increasing.

According to a 2020 GSMA report titled COVID-19 Network Traffic Surge Isn’t Impacting Environment Confirm Telecom Operators⁹⁹9https://tinyurl.com/4kva7dmb%https://www.gsma.com/about-us/regions/europe/latest-news-2/covid-19-network-traffic-surge-isnt-impacting-environment-confirm-telecom-operators/, the energy proportionality of the Internet is relatively low. For example, UK operator BT reported a 100% increase in daytime traffic across its fixed broadband network during COVID-19 times. However, this did not lead to a noticeable increase in electricity use or carbon emissions.

Reference [2] reports on the overlooked environmental footprint of increasing Internet use and states that, “depending on the energy supply mix and use efficiency, Internet traffic contributes differently to negative environmental impacts and climate change”. The report also claims that “as the number of Internet users increases, the number of online services and applications they use grow” [2]. This trend exacerbated the environmental footprint of the Internet, which can be observed through increased usage of Internet services such as video streaming (see section 3.3.3).

The accounting of carbon emission should distinguish between attributional and consequential emissions¹⁰¹⁰10https://hubblo.org/blog/attributional-vs-consequential/. Attributional accounting estimates the share of the total impacts of a system that can be attributed to a specific function. For instance: what is the share of the energy spent in the Internet for video streaming. Consequential accounting on the other hand considers the environmental impact induced by the implementation of this function. It compares a world without this function (say, without video streaming, where you drive your car to the movie theater) with a world with this function (say, with video steaming, where you watch a movie from home).

Traditional attributional accounting is often applied in reporting the carbon emission impact of an isolated unit, such as a company or specific sector. However, energy saving and carbon reduction measures in one unit or sector could lead to an overall increase in the larger socio-economic system, for example when a data center switches to renewable energy, while the overall amount of available renewable energy does not change, so that other consumption, with greater flexibility potential, would then use less carbon-neutral energy.

Details can be found in the presentation summarized in section 4.1.

There are different potential energy efficiency improvements, such as more efficient link layer communication, load aggregation and system consolidation, utilization-proportional energy consumption, load-adaptive system designs, and shifting workloads in ways that optimize energy efficiency, for example by concentrating workloads on fewer systems.

These general energy efficiency improvements can generally lower the overall energy consumption over time, for example per year, and they can increase energy flexibility potentials by offering higher levels of adaptability. However, increased levels of adaptability and myopic optimization can also lead to problematic flexibility requirements imposed on the energy infrastructure, for example when workloads and energy needs are concentrated, leading to unwanted spikes under a certain energy provisioning contract.

For an effective holistic optimization, it would be important to match energy requirements and saving potentials, for example through a mediator. As reported by Brown et al. [3] and as illustrated in Figure 1, it is estimated that, in Europe, sector-coupling flexibility and transmission can reduce total system costs by 37%.

In general, predictability is considered to be very valuable for the power grid. Optimizations that are not expected by the power grid and that are unaware of it can have a negative impact. Such local optimizations can create an undesirable need for increased flexibility from the power grid. It is deemed better to have sensitive optimization informed by a holistic system view.

In the talk described in section 5.1, an example was given, where an optimization significantly reduced energy, but as a result, the power price increased significantly. This is because temporary power surges are considered to have a particularly bad impact on the grid; in the short term, avoiding them, and “flattening” the use may sometimes be more valuable than reducing the average energy usage. Nevertheless, in the long term, it is an important goal to reduce the average.

Te-Yuan Huang (Netflix), also see section 4, presented facts on Carbon Emission and Streaming at Netflix. According to measurements at Netflix, 89% of the energy in the streaming pipeline is consumed by user devices (TVs and mobile viewing platforms, as well as home routers and peripherals), 10% is consumed by the core and access network, and only 1% by data centers. However the whole streaming pipeline only consumes about 3% to 5%, according to a recent Netflix Environmental Social Governance Report [4]; a breakdown of what these 3-5% consist of is shown in Figure 2.

In more absolute terms, in the UK, the carbon footprint of one hour of video streaming equates approximately 55 gCO2e (grams of carbon dioxide equivalents), which is roughly equivalent to three boils in an electric kettle [5, 6]. Details can be found in the presentation summarized in section 4.2.

Farzad Tashtarian from the Department of Information Technology, Alpen-Adria-Universität, Klagenfurt, Austria, presented Coconut [7], a COntent COnsumption eNergy measUrement daTaset for adaptive video streaming collected through a digital multimeter on various types of client devices, such as laptop and smartphone, streaming MPEG-DASH segments. The research in the paper presents perspectives on the energy usage of streaming devices¹¹¹¹11Dataset available at https://athena.itec.aau.at/coconut/.

With respect to assessing the power consumption of the streaming pipeline, the following research questions were proposed (see section 4.3):

• $\mathrm{■}$

  How can we accurately measure the energy consumption of video streaming components in real-world scenarios?

• $\mathrm{■}$

  To what extent can Generative AI impact the energy consumption of a video streaming pipeline?

• $\mathrm{■}$

  What is the effect of emerging protocols (e.g., QUIC) on energy consumption?

• $\mathrm{■}$

  How can we address the energy consumption challenges of immersive media?

• $\mathrm{■}$

  How can we balance latency, QoE, and energy efficiency effectively?

• $\mathrm{■}$

  What strategies can be employed to characterize and potentially improve the energy-efficiency of Adaptive Bitrate (ABR) algorithms?

• $\mathrm{■}$

  Can caching strategies be used to improve energy-efficiency?

Most of the energy optimization solutions on the mobile network side are dealing with temporarily shutting down nodes/sites or moving major workloads across sites and segments of the network on the system level. This has a strongly interlinked relationship with grid dynamics on the power system side.

Information exchange and a feedback loop between the two systems (mobile network and power grid) can be helpful in studying the impact of optimizations (on the network side) on power system dynamics on the grid. In some cases (small network sites), the impact could be minimal (hopefully) but in case of planning large workload deployments across the system, the impact can be significant. After understanding the potential impact of network side optimizations on the grid, further autonomous, fine-grained optimizations on the site/cluster/data center level can be made to make the cloud infrastructures and orchestration/scheduling systems more energy efficient. Details can be found in the presentation summarized in section 3.1.

There is significant potential to make AI, both training and inference, more efficient in itself. The seminar also discussed the need of making AI systems aware of and and responsive to larger system constraints.

Much of the focus in networking for AI workloads has been on making machine learning faster, without much consideration for energy efficiency, which is in part explained by the lack of data to assess emission impact of different training models. Hence, providing more comprehensive instrumentation of networking devices and data center equipment will be an important enabler to make AI workload placement more energy-efficient.

AI workloads generally also involve streaming of various types of telemetry data that is used to train and apply AI models from the sources of that data to where it can be processed. Some of the lessons from the greening of video streaming thus potentially also apply to the greening of the handling of AI data, both input and output, for inference usage, despite some obvious differences such as the number of clients to stream to (few in the case of AI processing, many in the case of video) or the direction of streaming (potentially from the edge to the core unlike from the core to the edge as with video). However, it may make sense to apply concepts from video such as DASH (Dynamic Adaptive Streaming over HTTP) and application of different video codecs (to send images at different resolutions, frame rates, and color depth depending on current network conditions) also to the streaming of AI data.

One approach to minimize energy consumption by AI workloads consists of moving workloads close to the sources of energy. One example is a project in Germany that aims to host compute nodes and mini-data centers in windmills. As 1/3 of energy is lost in the energy grid, moving the workload to the source will make AI workloads correspondingly more energy-efficient. This requires the ability to adapt workloads to available energy and renewable energy source dynamics, as well as applications that are flexible in terms of their workload processing demands. Similarly, the concepts of edge intelligence and federated learning can play an important role in allowing AI workloads that can be run across a network more energy efficient. Traditionally applied to minimize latency (by processing data close to the source) and improve privacy (by not requiring data to leave the edge), this provides for a promising additional incentive to be considered.

This discussion started from the joint optimization of the power grid and ICT, as increasing ICT energy efficiency can affect the grid’s proper operation. This means that ICT needs to consider power grid constraints and multi-disciplinary efforts in joint optimization may be required to achieve meaningful improvements. A second key take-away is that visibility about demand cycle, peak times and pricing models would be necessary for data center operators to consider. This would allow to match traffic workload forecasts with energy forecasts. On the other hand, it may be possible to use power in off-peak hours to scavenge work-loads in DC.

An additional discussion considers the optimal use of resources. It starts with scheduling in the operating system and orchestration layer, and extends to the scheduling of resources from an existing resource pool. For example, is it better to use one CPU at high utilization, or to use more CPUs in lower utilization? One observation from telecommunication operators is that CPU offloading is happening more and more, with GPUs and smartNICs in use. How to make these offload targets more energy efficient or use software replacement remains a challenge. It is also clear that the right hardware needs to be identified for different tasks, such as the optimal selection of access network technology. For instance, passive optical network (PON) can reduce energy in transport, but at the same time using optics can result in energy waste if used for short distance.

The discussions emphasized concerns that reduction in power usage and improvements in energy efficiency during operations do not necessarily imply greater sustainability. Instead, to optimize greenness, the parameter to minimize concerns the maximum power drawn at any one time. The reason is that the supply of power must be dimensioned to be able to satisfy that maximum power draw. It is this parameter, which determines the amount of electricity that needs to be supplied and hence generated, in turn correlated with carbon generation due to the inclusion of fossil fuels in the power mix. Reduction in power usage (on the scale of MWh) is thus beneficial only when planned sufficiently in advance for the grid provider to adjust to it. From the grid provider’s perspective, power that is not being put to productive use is effectively wasted as it still generated. In some cases, improvements in energy efficiency may even be counterproductive if reduction in energy use during some periods is compensated by additional spikes during peak periods.

One of the points brought up during the discussion concerns the role of prices as a mechanism to steer behavior of customers. Technical measures to improve environmental sustainability should be accompanied with price schemes to provide the right incentive structure to steer solutions in a way that truly reduces the amount of energy that is not just used but that needs to be supplied.

Also touched upon was the aspect of energy ratings of devices. Schemes such as Energy Star (used to allow for informed purchase decisions regarding the energy efficiency of electrical appliances) are not supported for networking equipment. While there are some questions regarding the efficacy of such schemes, ready availability of such data appears beneficial. To be meaningful, this would need to be provided in an objective manner. Today, it is not uncommon for data that is being advertised by equipment suppliers, or that is provided by equipment’s own power measurement instrumentation, to deviate significantly from data provided by objective measurements.

One of the areas already being explored is carbon-aware routing. However, insight from SCION’s work[4] will be hard to integrate with BGP. It is also clear that security should not be compromised, and it was asked if SCION multi-path will create more points of failure. Using energy labels [5, 6] was noted as an idea for sharing information that might work across domains. It was further noted that peering agreements can be used to improve the transparency of green path selection procedures. One of the questions is the hot potato routing problem. This is not a new problem, but the local optimization of carbon efficiency is an incentive for companies to pass packets as soon as possible to other ASes. Another aspect to consider is the actual distribution of traffic types in the backbone and potentially the need to differentiate traffic. From industry partners contributions, there appears to be significant differences between countries and networks, such as the existence or lack of video streaming caches within a network, which may lead to increased load. A takeaway is that sharing information across ASes is a big challenge, requiring validation that is not necessarily network based and potentially a third party to regulate or monitor. It was proposed that federations might be able to enforce it. Deciding on metrics that can be shared across domains remains an open research question.

Carbon aware routing and carbon aware TE potentially changes the location and severity of network congestion. Lower than Best Effort (LBE) is a type of traffic that can be less latency sensitive [7, 8]. LBE can be used for elastic traffic that can be treated more flexibly in the network (unlike streaming), e.g. to send it over a longer but more carbon efficient path. Recent work in CC considered pacing traffic, spreading it over time [9]. This conflicts with desirable wireless operation, where the objective is to send more traffic in bursts, in order to sleep more. There are some other conflicts, such as buffering, which can also introduce waste. There are important trade-offs: a need to provision, but also to avoid waste. A significant challenge in traffic engineering is how to benefit from renewable energy without exhausting its resources and requiring non-renewable energy to be used. There must be a control loop to guarantee this. On the other hand, previous work indicated that the power consumption of networking equipment is below the threshold to trigger such energy-source transition. A potential future CC challenge is AI traffic, which was already shown to be a bottleneck for frontier AI models training within data centers. However, it is unknown what the effect in wide area networks will be. Clearly, inference traffic differs from training and may look a lot like other types of network traffic. Furthermore, the AI tasks will affect the nature of the traffic: from object recognition from camera feeds to ChatGPT text prompts. It is also possible that new applications will require the use of training AI models over the wider network. Privacy-preserving A.I. training and inference schemes like federated learning and split learning may move load further out of the data centers into the WAN, necessitating resource allocation schemes sensitive to the energy usage patterns they affect.

The granularity of mobile and wireless communication is different from wired. For example, it is possible to turn off base stations at night or in certain city areas. It is also possible to turn off components within each base station, such as power amplifiers and antennas. However, the impact on transport and application layer would require further studies. There is a tension between what is ideal in wired, and what is idea in wireless (such as the previously mentioned pacing). Burstiness is good in wireless, but not in wired. Wireless networks may look at demand shifting and using the time dimension to cache contents; or shift mobile traffic onto wifi (offloading). A question was raised about the introduction of 6G, using smaller cells, higher frequencies. Would it be better to turn off smaller cells and use larger ones to cover with better efficiency? One power saving mechanism is to turn off equipment. In mobile, this ranged from the whole base station to only switching off the power amplifiers. Shutting down the whole base station may lead to aging concerns of the equipment with cooling down and powering up. Another important concern is management: once you turn something off, you have no visibility to it. Therefore, the management layer is afraid of not just losing visibility, but that once off, the resource won’t turn back on when required.

This section focuses on two separate yet related aspects of greening networking, sharing the fact that many overarching concepts in technology resemble cyclical characteristics when viewed from a holistic perspective. One of these aspects concerns control loops, i.e. the application of control logic to optimize environmental sustainability of solutions. The other aspect concerns the lifecycle of network equipment, which encompasses not only its operation but a larger cycle that also includes aspects such as manufacturing and decommissioning of equipment, all of which contribute to the environmental footprint and thus deserve consideration.

Three breakout groups were formed to discuss different aspects of these aspects: 1. Observations, 2. Planning and Forecasting, and 3. Hardware Lifecycle. The following sections present the outcomes of these conversations.

Control loops are a universal principle that plays a role in many solutions that aim to improve outcomes by continuously observing (the entity or domain that is being controlled and parameters that are being optimized), analyzing (to interpret and make sense of the observations, perhaps correlating them with other data), deciding (on which course of action to take in search of an improvement), and acting (i.e. performing the steps of the course of action).

Control loops also apply to many potential solutions that aim to improve green networking outcomes – for example, aiming to minimize the amount of energy used to provide communication services while meeting other important operational goals such as adhering to service level objectives or ensuring network resilience.

In the context of green networking, the following are important considerations to provide control loop components that will facilitate better solutions:

The natural starting point for greenness considerations involves the operation of a network, which involves the need to power equipment or to provide cooling and ventilation. However, greening the network does not start there – before equipment can be taken into operation, it needs to be manufactured. After equipment is decommissioned, it needs to be disposed of. In other words, there is a lifecycle for equipment that is to be considered. A substantial part of the green equation concerns the carbon that is embodied in a device itself, including energy use for manufacturing, not even to speak of use of land and water used to extract raw materials.

As a result, networks are greener if they can avoid deployment of hardware that is not really required. Today, networks are underloaded in many cases, meaning that the same service levels could arguably be achieved with less buildout [2, 3]. One reason for this is that overprovisioning is commonly used as a way to avert potential problems. Overprovisioning may be cost effective, but not necessarily “green”. Hence, a key question concerns when network upgrades are really required. By deferring network upgrades, embodied carbon can be amortized over longer periods of time, resulting in more environmentally-sustainable networks.

One common reason for equipment upgrades concerns greater energy efficiency of newer generations of hardware. In many cases, such upgrades can indeed lead to a positive impact on energy usage and resulting energy bills during network operations. However, they neglect the contribution of embodied carbon into overall sustainability. To have an effect, newer, more energy-efficient equipment needs to be in service for a certain amount of time for the operational carbon savings to offset the carbon that had been embodied in the decommissioned equipment and hence lead to a net-positive effective. This is illustrated in Figure 3. Holistic approaches to greening the network therefore need to take the entire lifecycle of network equipment into account, not only the operational phase during which pieces of network equipment are actually in use.

In wireless networks in particular, there are many reasons for hardware upgrades besides increasing the capacity (and capacity upgrades are the possibly unnecessary and hence not “green” element in this discussion). These include new abilities of modern equipment such as network slicing and being able to connect a larger number of customers in a small geographical area. In wired networks, reasons for upgrades are also manifold, as the discussion in Dagstuhl has shown:

• $\mathrm{■}$

  fiber does not have cross-talk;

• $\mathrm{■}$

  fiber is less sensitive to weather; rain affects the impedance of other wires, which makes it unusable when the snow melts;

• $\mathrm{■}$

  copper replaced by fiber can be sold back for a net profit;

• $\mathrm{■}$

  new hardware can offer new network features such as IPv6 support;

• $\mathrm{■}$

  regulatory reasons may require updates.

Addressing this begins with asking the right questions. Rather than trying to lobby against hardware upgrades, we should identify why updates are carried out: when are they done for capacity, and when are they done for features? It would be useful to gather more data about this (e.g., through network operator or end user surveys). Research should focus on studying (measuring, monitoring) the actual traffic demand and service to identify whether the running applications would (or would not) benefit from a capacity upgrade. This requires the development of novel measurement and monitoring tools that can enable both users and service providers to make much more informed decisions than “speed test” systems currently offer. This is in line with recent and ongoing work in the IP Performance Measurement (IPPM) Working Group of the IETF [4, 5].