HiPART: High-Performance Technology for Advanced Real-Time Systems

The recently released OpenMP6.0 [2] incorporates the taskgraph directive. This functionality implements reusable graphs of tasks to reduce the overhead of task orchestration (e.g., task creation) and minimise contention on shared resources (e.g., task ready queues). The functioning of taskgraphs is illustrated in Figure 3, with a sample code showing a taskgraph directive in Figure 3(a) and the execution flow depicted in Figure 3(b). Overall, the taskgraph construct encloses a region of code that can be captured as a TDG. Hence, it includes task-generating constructs (e.g., task and taskloop) that are executed whenever the region is reached and other statements (e.g., control-flow statements) that will only be executed when the region is recorded. When a taskgraph region is encountered at runtime, if a graph of tasks already exists for that region, it is played. Otherwise, or if the user explicitly asks for regenerating the graph through the additional graph_reset clause (e.g., when the condition within the clause in line 5 resolves to true), then the system generates the TDG of the region. How the TDG is generated and executed is implementation-defined, so it can either be generated statically, at compile-time, or dynamically, at run-time. In the latter case, it can be created while executing the region or in a preprocessing step to later execute the graph.

The taskgraph framework has already been tested using several HPC benchmarks including task and taskloop constructs. The results show speedups of up to 6x compared to the LLVM native implementation of tasking [34]. While upstream LLVM alreay implements a record and replay mechanism relying on runtime routines, a prototype implementation of the taskgraph construct including static (compile-time) and dynamic (run-time) recording capabilities is publicly available in https://gitlab.bsc.es/ppc-bsc/software/llvm-taskgraph.

Graph-based computation has recently become popular as it allows reducing the overheads of task orchestration in CPUs [34] and enhancing the performance of CPU-GPU heterogeneous systems using CUDA graphs (for NVIDIA devices) or HIP graphs (for AMD devices). Leveraging the similarities between taskgraphs and CUDA and HIP graphs, an extension of the OpenMP taskgraph framework has been proposed to deploy OpenMP taskgraphs in GPUs using the GPU native graphs [35], i.e., CUDA or HIP graphs. The results show similar or better performance compared to original target tasks exploiting the target threading model due to the reduction of kernel offloads and an optimised orchestration of the tasks. Furthermore, the framework also shows better scalability with the number of processors.

Adaptability and performance portability in complex and changing heterogeneous systems cannot only be accomplished through compiler optimisations or runtime mechanisms. OpenMP includes features to define function specialisations. In the standard, different user functions can be linked together through the declare variant directive, which establishes different functions to implement a unique functionality and the condition that the compiler must check to statically decide which implementation is used in each function call. Although this presents a step forward, adaptability can only be obtained at compilation time. Therefore, runtime changes (e.g., a permanent failure in a device) cannot be considered.

HiPART relies on an extended interpretation of OpenMP variants that gathers metrics at run-time to dynamically decide among the set of function specialisations provided by the user [15]. This procedure allows for considering the system’s dynamic conditions, like workload and energy consumption. To that end, the compiler follows the flow shown in Figure 4, producing two distinct binaries: (1) an instrumented version equipped with runtime calls that collect metrics about resource usage and (2) a version that interprets the metrics gathered by the instrumented version to guide variant selection. The second binary is generated only after the instrumented binary runs and collects the metrics.

The metrics captured during warm-up include average and peak CPU usage (%), average and peak GPU usage (%), thread stack memory consumption (%), heap memory consumption (%), GPU memory consumption (%), and execution time (ms). During the execution of the final binary, users can provide a list of metrics to guide variant selection, represented as a set of comma separated triplets of the form of metric:threshold:weight, where metric is $cpu,gpu,mem,stack$ or $heap$, threashold indicates a percentage that, when reached, forces the runtime to select the variant that stresses less the corresponding metric, and weight is an optional parameter that indicates the weight of the metric.

The evaluation of this proposal [15] shows the capability of the system to tune the optimal number of threads for each parallel region, prevent errors or slowdowns in systems with limited memory, and effectively swap between CPU and GPU implementations when one resource is overloaded. However, the method might introduce unbearable overheads in systems with rapid fluctuations, and decisions might soon be outdated. HiPART plans to mitigate these effects with mechanisms like splitting functions. Furthermore, other planned extensions include more refined predictive models that can foresee system state changes and new metrics like power consumption.

The dynamic nature of the OpenMP task scheduler and the use of work-stealing to balanc the workload of all threads introduce uncertainty in the execution and, although good-enough in general terms, entail certain overheads. Recent works have proposed the use of temporal conditions to derive a more efficient task-to-thread mapping able to reduce system response time and running time and providing better predictability [26]. The scheduling mechanism proposed is depicted in Figure 5. It is split into two phases: (1) allocation, assigning each task to a thread, and (2) dispatching, selecting a task from the ready task queue. A series of heuristics are proposed for each of these phases, leveraging information about the number of tasks in a thread’s ready queue and their execution time, among other aspects.

An evaluation of the proposed heuristics compares to common implementations in OpenMP runtime, including breadth-first scheduler (BFS) and work-first scheduler (WFS), regarding response time and running time. The results show that the response time produced by some heuristics is lower than the default LLVM scheduler in most cases, and the variability in the results given by the heuristics is lower than that of the default scheduler.

HiPART plans to extend this work by providing a schedulability analysis of state-of-the-art mapping strategies and the suggested heuristics in relevant applications. Furthermore, the project considers extending these mechanisms to heterogeneous systems with multiple GPUs.

Several CPSs are sensible to transient faults due to the harm these may cause to the safety of people, the environment and the system itself. Unfortunately, models like OpenMP lack the mechanisms to provide fail-operational behaviour. Accordingly, HiPART is developing fault-tolerance techniques to enable software-based task-level replication and user-directed fault detection to mitigate the impact of transient faults.

The proposed fault-tolerance mechanism is based on an extension to OpenMP to define task replication. Figure 6 shows an example of such an extension. The sample code in Figure 6(a) illustrates the syntax proposed (lines 9 to 11) [17], where:

• $\mathrm{■}$

  the clause replicated specifies the number of replicas to create through an integer expression ¿ 1 (3 in the example) that indicates the total number of tasks to be generated (including the original instance and the replicated instances), and the replication strategy through the spatial, temporal, or spatial_temporal optional keywords. Spatial replication forces each task to be executed on a different resource, such as a processor core or an architecture, allowing them to run in parallel; temporal replication ensures that tasks execute one after the other, enforcing sequential execution; and spatial_temporal replication combines both approaches, requiring tasks to run on different resources while also executing sequentially. If no replication type is specified, the default behaviour allows the tasks to run in parallel without restrictions, thus favouring performance.

• $\mathrm{■}$

  the clause replica_private defines variables that will be replicated for each task replica, i.e., each task will have its own copy of the variable. By making data private to replicas, this clause prevents data races and ensures each replica operates independently without causing inconsistencies in shared data.

• $\mathrm{■}$

  the clause replica_firstprivate specifies a list of variables that must be firstprivate to each replica (i.e., each task will allocate a new space and initialize its value to the value of the original variable at the time the task is instantiated). The compiler performs a shallow copy unless a shaping expression (e.g., $[size]a$, an array $a$ of $size$ elements) is defined, in which case it performs a deep copy considering the corresponding size. This clause ensures that the replicas start with a consistent state, improving fault tolerance while maintaining independent execution for each replica.

The execution flow in Figure 6(b) illustrates the behaviour of the replication mechanism. The thread that encounters the task replicated (line 8) creates a replication set with three tasks: the original and two replicas. Since the spatial constraint is specified, the OpenMP runtime system prevents threads from executing multiple tasks within the same replication set. Threads can further be bound to cores through the OMP_PROC_BIND environment variable for the whole execution or the proc_bind clause, attached to a parallel construct.

Figure 7 shows preliminary results of the impact of using task replication in the Barcelona OpenMP Task Suite (BOTS) [8], a benchmark suite with eleven benchmarks exposing different memory profiles and CPU consumption when randomly inserting a single bitflip per execution in either memory or registers. The result of a bitflip can be a benign fault, when the application completes with the correct result without detecting any error, an output error when the application finishes normally but the output is incorrect, a crash, when the application terminates unexpectedly due to an internal error, or timeout, when the application hangs and does not complete. The results compare a bitflip in the sequential vanilla version, namely vanilla, with a bitflip in the replicated version, namely GuOMP. Considering that bitflips typically occur in the stack in applications with minimal memory usage, examples like Floorplan, Alignment, Fib, Nqueens, and Knapsack exhibit high tolerance to faults because the majority of the stack is not actively used. Oppositely, benchmarks like UTS, which uses about 1.5MB of stack memory, are more prone to crashes. On the other hand, applications with higher use of dynamic memory see more significant effects from bitflips. Still, this behaviour depends on the specific algorithms and their memory access patterns. For example, a single bitflip in the Sort integer array disrupts the output, as sorting algorithms depend on precise memory operations.

HiPART plans to extend the proposal for fault-tolerance in OpenMP with a user-defined consensus and voting mechanism that will provide better accuracy when comparing results from different replicas. Furthermore, replicas will be extended to exploit function variants to boost resilience in heterogeneous systems. Finally, to provide fault-recovery capabilities, HiPART plans to enable communication between the runtime system and the application through runtime routines and OpenMP data structures and offer a checkpointing mechanism to store selected memory objects. These methods combined will enable users to handle errors in the most adequate way for each application.

The throughput-centric design of GPUs poses challenges when integrating them into time-sensitive CPS. Modern systems recently evolved to minimize overheads and interference along the critical path through advanced mechanisms, such as CUDA graphs in NVIDIA devices and HIP graphs in AMD devices. However, GPU vendors provide ecosystems specific to their products, preventing code portability. HiPART has integrated event-based synchronizations into OpenMP and extended the support for OpenMP taskgraph to CUDA/HIP graphs to notably reduce interference and overheads in time-sensitive applications.

Figure 8 shows an example of the proposal for event-based synchronization in OpenMP. Figure 8(a) depicts an example of generating four interdependent target tasks, two synchronized with events. The corresponding TDG is shown in Figure 8(b). Upon the encountering of an attach clause, the implementation creates a new allow-launch event and connects it to the beginning of the execution of the associated task region. The generated task can only start executing its associated structured block when the allow-launch event is fulfilled. This will happen when another thread encounters the fulfill_event directive with either the cpu_notify or gpu_notify clauses taking the same event as the argument.

Preliminary experiments measure the response time and time variability of the Adaptive Optics (AO) real-time controller illustrated in Figure 9. The application combines two functionalities: (1) a pixel processing method that takes raw data from wavefront sensor cameras and processes the pixels with a series of arithmetic kernels, and (2) a series of matrix-vector multiplications that produce a command as a vector sent to the deformable mirror actuators of the physical component, typically a telescope.

Figure 10 shows the response time of the AO application using CUDA (in Figure 10(a)) and HIP (in Figure 10(b)). In NVIDIA devices, our implementation (OpenMP CUDA Graph & GPU sync) achieves 16$\mu s$ max. jitter, with 538$\mu s$ mean execution time (MET) and 554$\mu s$ maximum measured execution time (MMET). Although the response time is slightly higher in our OpenMP CUDA version than the native solution (Regular CUDA Graph & GPU sync), the max jitter is identical, showing comparable predictability. In AMD devices, however, the OpenMP HIP version (OpenMP HIP Graph & GPU sync) achieves a slightly higher max. jitter of 21$\mu s$ (443$\mu s$ MET and 464$\mu s$ MMET). As expected, the third method using the CPU synchronization strategy showed the largest execution times and a max jitter of 28$\mu s$ (562$\mu s$ MET and 590$\mu s$ MMET) with CUDA and 33$\mu s$ (478$\mu s$ MET and 511$\mu s$ MMET) with HIP. All in all, our proposal delivers MET and jitter comparable to the native CUDA/HIP implementations while maintaining a simple, directive-based programming style.