Robust Query Processing in the Cloud

The robustness of database systems is an important and extensively discussed topic in academia and industry. While various new approaches have been proposed to improve the robustness of database systems – including new robust query operators such as smooth scan – surprisingly, there is still no method that allows us to holistically analyze and quantify the robustness of a database system. In this paper, we define the very first method that allows us to quantify the robustness of a database system. Intuitively, our method analyzes whether small changes in the execution of a workload lead to only small observed changes in the performance of the database system. Building on this robustness definition, we propose a new benchmark framework that tests the robustness of a database system. This framework can take any existing workload as input (e.g., TPC-H or TPC-DS) and by injecting small perturbations targeting, for instance, query and data characteristics, quantify the robustness of a system. Importantly, along with the benchmark, we propose a new robustness metric that allows us to quantify robustness at the system level and enable comparison across systems or versions of the same system.

We propose the concept of performance SLAs (Service Level Agreements) for SQL workloads, as an enabler for new pricing models in cloud databases. We identify two new research areas necessary for making this possible: (i) reliable methods to determine SLA pricing and associated financial risks and profits, (ii) technical innovations in cloud database engines that take into account query workload deadlines, and employ elastic resource allocation to make these deadlines. In all, we think that this new model, on the one hand finally offers customers a real handle on workload SLAs, while on the other hand enables both customer and provider cost saving as well as reduced hardware resource usage (and thus reduced carbon footprint).

In today’s cloud analytics market, customers choose providers to serve their query workloads for an extended period of time. In contrast, an efficient choice per query (or per workload) would enable and encourage innovation for efficient database systems. Building on shared storage in the public cloud, we propose the concept of a query exchange. A query exchange implements a competitive market, brokering queries to the provider with the lowest bid for execution satisfying the customer requirements. Using open table formats and appropriate access governance, alternative providers can read and process the same data sources. Relying on specifications of data, metadata and workload SLAs, the query exchange aligns customers’ incentive to lower costs with providers’ incentive to process queries for which they have a competitive advantage, thus achieving efficient, scalable, and robust query performance in the cloud.

The Dagstuhl Seminar “Database Indexing and Query Processing” in March 2022 produced the SPA paper [1], which presented a general framework for improving the performance of scanning modern column stores. SPA introduced several interesting building blocks, such as incremental building of intermediate structures and using economic principles to guide the construction (and deconstruction) of such structures. We explored several methods inspired by that framework to improve the performance of query processing in a robust way. More precisely, we explored the following ideas, in a new framefork SAUNA – Saved Aggregates Unleash Novel Acceleration.

A major challenge in distributed query processing in the cloud is the handling of mispredicted input cardinalities and skew. In order to mitigate this problem, our group set out to devise graceful strategies for unexpected out-of-memory situations in distributed joins, group-bys, and window aggregates. After detailed discussions of existing system architectures and operational challenges, we defined a framework for detecting and mitigating such out-of-memory situations. At its core, we create virtual partitions, and optimize configurations including the mapping of virtual partitions to nodes, build-side broadcasting, probe-side materialization, as well as the number of utilized nodes. Furthermore, our group explored a broad list and classification of other unexpected performance problems of user-facing issues (e.g., LIKE predicates, UDFs, and APIs), query compilation issues (e.g., generated huge queries and expression trees), as well as runtime challenges (e.g., parallelism for pipelines with small inputs, loads, tail latency, resource isolation, large strings).

Many modern cloud database systems use disaggregated architectures, separating the computations and the underlying object storages (e.g S3). Traditionally, database systems have used caches to improve performance to improve the data on local SSDs, essentially following the so called 5-minute rule of thumb to decide when to cache in-memory or read directly from local storage. As data moves to object stores, which have highly unpredictable tail latency and explicit costs per access, the question naturally arises whether these new disaggregated architectures need comparable caches. In practice, many systems do use caches between object storage and compute, however caches can introduce new overall system robustness challenges (e.g. cache misses) as well as add non-trivial expense both in terms of engineering and operational overhead. In this paper, we review the requirements that lead to object store caching layers, analyze the design space, and propose new rules of thumb to help system designers determine under what circumstances they should introduce caches instead of reading directly from cloud object storage.