Power, Energy, and Carbon-Aware Computing on Heterogeneous Systems (PEACHES)

Heterogeneous compute architectures are key to energy savings, provided that software chooses to components of software systems on the individually best suited hardware components available. High-performance embedded devices and other battery-powered devices from mobile phones to laptops are typically equipped with various instances of multiple types of CPU and GPU cores with highly different performance/energy trade-offs and characteristics. The open question is how applications can be mapped to such architectures under global objectives such as meeting deadlines or achieving required performance levels while using the minimal amount of energy possible.

The TeamPlay coordination language is geared towards describing componentised applications, their dependencies, constraints, and objectives on a high level of abstraction. As a coordination language TeamPlay delegates the implementation of software components of reasonable size and complexity to established programming languages for programming-in-the-small and entirely focusses on coordination aspects such as time-wise and spatial scheduling, dependencies and communication, etc, all in line with the aforementioned global objectives. Compiler and runtime system middleware are charged with establishing transparency about the inevitable trade-offs and actually run applications on the heterogeneously parallel system architectures targeted.

In the quest for sustainable and efficient software, accurate energy consumption measurement is paramount. This talk addresses common pitfalls and best practices in green coding, specifically focusing on the importance of understanding the tools and techniques used for energy measurement in software development. Through real-world examples and critical analysis, we explore the limitations of popular measurement tools like Intel RAPL and the Gude Power Meter, and discuss the necessity of consistent temperature conditions, disabling CPU features like Turbo Boost and Hyper-Threading, and the challenges of using CPU utilization as a reliable metric. Moreover, we emphasize the significance of validating machines for anomalies, automating processes, and considering the overhead of measurement setups. Practical advice on using specialized operating systems, non-standard sampling intervals, and handling external network traffic is also provided. This talk invites discussion and collaboration on improving green coding practices, ultimately aiming to refine how we measure and reduce the environmental impact of software development.

It is widely accepted that energy proportionality, i.e., the ability to scale the power drawn by a system with its workload, is an enabler for energy efficiency. Putting parts of a system to sleep – turning them off or into some low-power mode – when they are not needed is the B-A-BA of proportionality. In networks, a simple part one can turn off is a link: if traffic volume is low, one can often route traffic away from some links that can be put to sleep to save energy. But how to select the links one can safely turn off without causing congestion elsewhere in the network? And how much energy would that actually save?

Our ongoing research addresses these questions. We found that, in lowly loaded networks, simple heuristics appear sufficient to decide which links to turn off. Using real topology and traffic data from two small ISPs (Switch (CH) and SURF (NL)), we found that about 1/3 of links can be turned off on average. As I will discuss in the talk, the conversion into expected energy savings requires further research.

Cloud servers can be designed to optimize for carbon emissions. Hardware developments such as CXL and dense/efficient CPU cores offer improved performance/Watt and the ability to give a second life to older components with high embodied carbon, such as DIMMs. The talk covers the design of such servers, and a framework for evaluating the carbon savings at the datacenter, including features such as application performance, customer adoption, maintenance implications, and workload packing implications.

This presentation explores the energy consumption associated with Information and Communication Technology (ICT) systems, with a particular focus on how software influences overall energy use. It proposes the introduction of energy labels for software and software modules. ICT’s contribution to global greenhouse gas (GHG) emissions is substantial, accounting for an estimated 1.8 % to 2.8 % of total emissions, primarily driven by the operational phases of these systems [2]; e.g., the GHG emissions was evaluated to be 1.4 % in 2020 for the operational phase [1]. Existing research demonstrates that optimization at the software level can lead to significant energy savings. For example, replacing specific JavaScript libraries in web applications can reduce energy consumption by up to 30 % [3]. Despite the potential for considerable energy savings, a large portion of software developers do not prioritize energy efficiency during development, with only about 10% attempting to measure energy consumption [4]. We propose the creation of energy labels for software, inspired by the EU energy labels for household products. These labels are intended to provide relevant, comparable, and accurate information on energy efficiency, thereby enabling users and developers to make informed decisions. The EU labels have been recognized as “one of the most cost-effective ways to enhance security of energy supply…” [5]. Our preliminary study on WordPress plugins – software modules – indicates that (1) it is feasible to assess individual software modules independently when investigating the energy consumption of web-based software, (2) it is possible to measure and determine their impact on the energy consumption of the entire system, and (3) this approach allows for the first comparison of energy consumption among individual modules offering similar functionalities [6]. The study suggests that it is feasible to develop energy labels for software in the future, providing relevant, comparable, and accurate information on energy efficiency. We envision creating energy labels for functionally equivalent modules and, in the long run, standardizing the process for creating such labels to be applicable across different levels within the software stack, effectively driving reductions in the environmental impact of software.

In this work, we present an investigation into 10 years of GCC versions from an energy perspective. Specifically, we employ Intel’s Running Average Power Limit (RAPL) technology to compare the energy consumption of different versions of GCC when they compile a relevant subset of the SPEC benchmark. Initial results show no significant improvement and may suggest that older GCC versions are more efficient at compiling. However, more measurements are needed to obtain a clear statistical result.

Memory subsystems have long been overlooked consumers of operational energy in computing systems. Novel memory technologies, like NVRAM, further introduce wear-and-tear behaviour, making embodied carbon a factor in memory placement decisions.

With cMemento we aim to supply system software with knowledge about workload behaviour and hardware resources alike for more carbon-aware steering of allocations.

One key requirement is an understanding of a workload’s sub-page memory access patterns. Hence, we propose Heimdall, a novel memory device that occupies part of the physical address pace. It is backed by other DRAM in the system but can snoop on and redirect requests on the memory bus. Our prototype runs on an AMD Zynq 7000 SOC, with an ARM Cortex-A and tightly integrated Xilinx FPGA.

Heimdallr can simulate heterogenous memory devices, using the FPGA’s block RAM. We present an architecture and operating system interface to trace and process access patterns on the memory bus, without impeding the overall bandwidth. Although we add about 90% latency while the tracing is enabled, an early evaluation shows that we can use the gained knowledge to move the most promising allocations to the block RAM – saving an average of 11% in operational energy permanently.

This work is funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Individual Research Grant 502228341 (Memento) as part of the Priority Programme on Disruptive Memory Technologies (SPP 2377).

Software-related CO2 emissions from the information and communications technology sector currently account for 2.1%–3.9% of global emissions. With the latest advancements in Machine Learning (ML) systems, this percentage of global emissions is estimated to increase. This raises the question: how to create more awareness on the sustainability impact of training and inference of ML models? In this talk, I navigate through the capabilities of the GAISSALabel tool [1] form the GAISSA project [2]. The tool shows how to report green ML metrics, to measure and assess the energy efficiency in ML systems, and shows recommendations on how to reduce their carbon emissions (e.g., reducing ML model size, using lightweight architectures, early stopping, optimization tactics as quantization and pruning, energy-efficient ML serving, and when to deploy in cloud or edge). Efforts to standardize the reporting of ML carbon emissions are needed (e.g., ISO 20226, SCI from GSF). Besides reporting, the aggregation of the results of empirical studies from the green software engineering community, in order to create theories with consolidated and quantified software tactics to reduce emissions, is key.

The proliferation of data-intensive workloads has led to a significant increase in the volume of data that computing systems need to handle. The growing significance of memory as a bottleneck has resulted in increased total cost of ownership (TCO) for servers. Existing methods to enhance bandwidth involve raising the data rate; while scaling up capacity is achieved through techniques like 2DPC or increasing DRAM density, each of which are coming at the expense of the other. An alternative approach is to scale out by adding more CPUs or through 3D stacking memory packaging technology, but these approaches come at the expense of much higher TCO.

In this talk I will discuss the potential use cases of Compute Express Link (CXL) technology. CXL is an open industry standard interconnect offering coherency and memory semantics using high bandwidth, low latency connectivity between processors, accelerators, memory, storage, and other IO devices. It helps address the above challenges by scaling up through increased bandwidth and capacity per core in the system. CXL is gaining faster adoption because it introduces load/store semantics to PCI Express (PCIe) physical layer, enabling expansion of both capacity and bandwidth at access latency comparable to remote non uniform memory access (NUMA) node DRAM memory. This addresses the CPU pin count challenge and avoids the TCO associated with adding new servers and processors just to get additional bandwidth and capacity.

Until recently, the emissions caused by storage resources were largely disregarded, with both academic and industry efforts having largely focused on emissions attributed to compute resources. To keep up with the excessive demand for storage resources across disciplines and industries however, all available opportunities need to be explored to enable data volumes to grow more sustainably. While research efforts on reducing storage emissions are still in an early stage, they largely focus on three general strategies to reduce storage emissions: By lowering and/or shifting power, by increasing the density of storage devices, or by extending the lifetime of storage hardware [1].

With the development of the UltiHash object storage, we are proposing an additional strategy to reduce storage emissions. We propose that using fine-grained deduplication, we can reduce the number of storage devices required to store data. With this approach, we are accepting slightly increased operational emissions caused by spending CPU time once to reduce both operational and embodied emissions of storage devices over the retention time of the stored data.

Our approach differs from existing deduplication mechanisms in two distinctive ways: first, using a variable fragment size enables us to detect more fine grained occurrences of redundancies. Second, instead of operating on a limited scope such as s single file system or a single file server, the scope of deduplication spans across a distributed storage cluster, aiming even towards federations of storage clusters.

While deduplication rates of our approach highly depend on the properties of the dataset to be deduplicated, initial tests have yielded data deduplication rates of up to 70% for uncompressed datasets, and even deduplication rates >= 10% for compressed media files.

In the security community it is commonly assumed that performance and energy are correlated and it is sufficient to just measure performance overheads to assess the overall costs of a security measure. We investigated whether this is true for the Linux kernel and found that in a significant number of cases energy and performance costs are actually not correlated. We conclude that to assess the energy costs of security, it is currently unavoidable to perform actual energy measurements.

The integration of Artificial Intelligence (AI) into software systems has significantly enhanced their functional capabilities, albeit at the cost of increased energy demands. Ensemble learning, which combines predictions from multiple models to derive a single output, exacerbates this issue by intensifying cumulative energy consumption. We address these trade-off challenges through an industrial case study of an AI-enabled system using ensemble learning for document information extraction. Our findings demonstrate that our approach of selecting a subset of models by optimising the GreenQuotientIndex, which accounts for the trade-off between accuracy and resource usage, can lead to resource savings of up to 99 %. Additionally, this approach reveals unexpected gaps in system optimisation related to accuracy. Selecting a subset of AI models can even increase accuracy by up to a factor of 2.7. These insights underscore the necessity of defining sustainability as a critical requirement in software development, rather than defaulting to a higher number of models without careful consideration.

Cyber-physical energy systems (CPES) are digitally controlled systems in which the degree of heterogeneity and complexity has become so high that their particular behavior is no longer necessarily transparent – not even to developers or users with technical background. One way of providing more transparency is by giving an explanation of the particular behavior of interest; and for that, the subsystem in question is in the best positions, as it best could explain itself. So that a self-explanation is operationizable by its addressee, it needs to be geared towards that specific addressee. Also, it has to be provided timely and in a context-sensitive fashion, and in particular at a level of abstraction different from the behavior that should be explained.

Consider as example a wind turbine controller. Characteristic for controllers is the feedback loop, in which the input depends non-trivially on the output as well as the interaction with other control systems, e.g., the controller for the collective pitch angle interacts with the controller for the yaw. An addressee of an explanation of that subsystem might be a maintainer or an optimizer or fault searcher as well as an operator of a client system. Now assume the addressee observes an unexpected output value and thus is in need of an explanation. So that the addressee can properly respond, it must not consider one such parameter just in isolation. On the other hand, it cannot be expected to understand the full control-theoretic details of its dependences. Instead, the addressee needs a simplified, abstract explanation of the system behavior, here: of the turbine controller.

For reducing the complexity of control software while keeping mutual dependencies of the feedback loop, we exploit that the differential equations of the control problem can be piecewise approximated by simple (linear) functions and that in an online setting those simplifying “control patterns” can be swapped out when outdated. The overall soundness of the approximation then depends on the “control patterns”, the choice of time windows, and the safe and timely swapping of patterns.

The talk motivates the abstract explanation pattern introduced by the CAUSE (Concepts and Algorithms for–and Usage of–Self-Explaining Systems) Research Training Group and, for wind turbines, illustrates this explanation pattern on its control software where explanations are executable networks of timed automata.

Improving the efficiency of software means running more computing using less resources, and energy. However, in order to achieve this goal we need to first be able to quantify the energy consumption of applications. Quantification is hard for multiple reasons: in realistic environments multiple applications run on shared infrastructure. First, there is no way to measure the energy for individual applications, hence we need to approximate it. Second, in many cases there is no access to the actual on-line power reads of the machine (e.g., in cloud environment). Third, applications behave differently on different hardware and configuration leading to combinatorial explosion. In this talk I will describe the approach we took with our open source tool Kepler to address these challenges.

Non-volatile RAM (NVRAM) is fast, byte-addressable, non-volatile memory that can replace traditional DRAM in systems. Doing so improves the non-functional properties of a system, including lower energy usage, faster storage and higher resiliency. This talk focuses on the energy usage aspect.

There are several opportunities to save energy by using NVRAM. First, opposed to DRAM, NVRAM does not need to be refreshed, leading to low idle power consumption. Second, by getting rid of all volatile memory in our systems, they become able to rapidly power off and resume their operation. This can be used to shut off machines during even short periods of idle time, while staying able to quickly respond to requests. Third, a lot of persistency mechanisms in operating systems are no longer necessary, as NVRAM provides that persistence for free. This means we can declutter our operating systems, getting rid of energy intensive I/O operations in the process. However, there are still challenges to overcome to realize those savings. Today’s systems are built with DRAM in mind. Booting into NVRAM is challenging and slow firmware hinders rapid suspend/resume mechanisms. Finally, introducing a new class of memory into a system requires the operating system to perform placement decisions to ensure the heterogeneous memory resources are used most efficiently.

This work is funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Individual Research Grant 465958100 (NEON).