Abstract 1 Introduction 2 Preliminaries 3 Composing NIIS protocols 4 Reductions and extension-based proofs 5 Conclusions References

A General Class of Reductions and Extension-Based Proofs

Yusong Shi ORCID Department of Computer Science and Technology, Tsinghua University, Beijing, China Weidong Liu ORCID Department of Computer Science and Technology, Tsinghua University, Beijing, China
Zhongguancun Laboratory, Beijing, China
Abstract

The concept of extension-based proofs models the idea of a valency argument which is widely used in distributed computing. Extension-based proofs have been shown to be limited in power: there is no extension-based proof of the impossibility of a wait-free protocol for (n,k)-set agreement among n>k2 processes.

Previous work used a restricted class of reductions to show that there are no extension-based proofs of the impossibility of wait-free protocols for some other distributed computing problems. It is known that for a restricted class of reductions, if a task 𝒯 reduces to 𝒮 and 𝒯 has an augmented extension-based proof that it is impossible to solve in the NIS model, then so does 𝒮. We introduce multiple-instance extension-based proofs and show that, if 𝒯 reduces to multiple instances of 𝒮, instead of just one instance and 𝒯 has an augmented extension-based proof, then 𝒮 has a multiple-instance extension-based proof that it is impossible to solve in the NIIS model. We introduce a new version of extension-based proofs that can further our understanding of extension-based proofs and their limitations.

Keywords and phrases:
Reductions, Impossibility proofs, Extension-based proof
Copyright and License:
[Uncaptioned image] © Yusong Shi and Weidong Liu; licensed under Creative Commons License CC-BY 4.0
2012 ACM Subject Classification:
Theory of computation Interactive proof systems
; Theory of computation Distributed algorithms ; Theory of computation Distributed computing models ; Theory of computation Problems, reductions and completeness
Acknowledgements:
We would like to thank Faith Ellen and Shihao Liu for helpful discussions and the anonymous reviewers for their comments.
Editors:
Silvia Bonomi, Letterio Galletta, Etienne Rivière, and Valerio Schiavoni

1 Introduction

In 1985, Fischer, Lynch, and Paterson [12] proved one of the most important results in distributed computing: There is no deterministic wait-free protocol for the consensus task in the asynchronous message passing system. The key idea of their proof is called a valency argument, which proves the existence of an infinite execution when the algorithm never outputs an incorrect output. The (n,k)-set agreement task, which is a generalization of the consensus task where n processes can output at most k different values, first proposed by Chaudhuri [11], was independently shown to have no wait-free protocol by Borowsky and Gafni [7], Herlihy and Shavit [14], and Saks and Zaharoglou [17]. These papers used topological structures to model tasks and protocols.

In [2, 3], Alistarh, Aspnes, Ellen, Gelashvili and Zhu pointed out the differences between valency arguments and combinatorial or topological techniques. In those proofs using combinatorial or topological techniques, the existence of a bad execution is proved, but not explicitly constructed. In the proof by Fischer, Lynch and Paterson, an infinite execution can be obtained by extending an initial execution infinitely often. Alistarh et al. generalized this type of proof and called it an extension-based proof. An extension-based proof is defined as an interaction between a prover and a protocol that claims to solve a task. Initially, the prover only knows the initial configurations of the protocol. To obtain information about the protocol, the prover proceeds in phases. In each phase, the prover submits queries to the protocol to learn information about this protocol. The prover wins if it discovers any error of the protocol. Otherwise, the protocol wins. If there exists a prover that can win against any protocol that claims to solve a task, we say that this task has an extension-based impossibility proof. The proof of the impossibility of consensus [12] is an example of an extension-based proof. It was shown that there are no extension-based proofs for the impossibility of a wait-free protocol for (n,k)-set agreement in the non-uniform iterated immediate snapshot (NIIS) model [2] and in the non-uniform iterated snapshot (NIS) model [3].

Do other tasks also have no extension-based impossibility proofs? One possible way to extend existing results is via reductions, which is a widely used technique in the field of theoretical computer science. The composition 𝒮 of two tasks 𝒮 and is a task that can be solved by a protocol that first solves 𝒮 and then solves , where outputs from 𝒮 serve as inputs to . A reduction from task 𝒯 to task 𝒮 is a protocol that solves 𝒯 by using a protocol that solves 𝒮. Processes may use many instances of 𝒮 and communicate with one another to decide the input values for each instance.

Brusse and Ellen [10] introduced augmented extension-based proofs, which provide an additional type of query, and gave the first result about reductions: if 𝒯 reduces to 𝒮, and 𝒯 has an augmented extension-based proof that it is impossible to solve in the NIS model, then so does 𝒮, when the reduction is limited to the use of only one instance of 𝒮. Such a reduction can be represented by 2𝒮1 where 1,2 are solvable tasks in the NIS model. Our goal is to answer the open question proposed in their paper, i.e., whether the extension to reductions that allow multiple instances of 𝒮 is possible. In other words, we will consider the reductions that have the form l+1𝒮l𝒮1 where 1,2l+1 are solvable tasks.

Suppose that there is a protocol that claims to solve the task 𝒮. To make use of the reduction from 𝒯 to 𝒮, we need to compose multiple copies of this protocol and the protocols that solve 1,2l+1 to generate a protocol that solves the task 𝒯. This is not straightforward as in uniform models, such as the IIS model, in which all processes terminate after the same number of rounds during all executions of a protocol.

This paper mainly concerns two topics addressed in [10]: a way to compose non-uniform protocols and the construction of an augmented extension-based prover 𝒫S for 𝒮 from an augmented extension-based prover 𝒫T for 𝒯. All the discussions in this paper are about the non-uniform iterated immediate snapshot (NIIS) model rather than the non-uniform iterated snapshot (NIS) model. The techniques in the two models are quite similar.

1.1 Our contribution

The way of composing multiple protocols in [10] can be generalized to a larger class of reductions. Note that these protocols are assumed to be transparent which means that the entity that tries to compose the protocols knows everything about these protocols. We present some properties of this composition that are necessary for discussions of the construction of an extension-based prover in both this paper and in [10], but are missing in the previous work. For example, a protocol in an extension-based proof is not a transparent protocol as assumed in the composition of multiple protocols, as the prover can only learn information about it by submitting queries to it. The usage of this composition in the discussions about extension-based proofs must be inspected in a more rigid way.

Next, we prove that if the task 𝒯 reduces to the task 𝒮 using multiple instances of 𝒮 and 𝒯 has an augmented extension-based proof that it is impossible to solve in the NIIS model, then 𝒮 has a multiple-instance extension-based proof that it is impossible to solve in the NIIS model. Our proof is not a trivial generalization of the proof in [10]. A technical difference here is that when the reduction from 𝒯 to 𝒮 involves more than one instance of 𝒮, we have to discuss a new version of extension-based proofs, called multiple-instance extension-based proofs, where the prover interacts with multiple instances of the protocol (instead of only one instance of the protocol in the original definition). Then, similar to the approach in [10], we describe how to construct a multiple-instance extension-based prover 𝒫S from an extension-based prover 𝒫T. Given a protocol that claims to solve the task 𝒮, the prover 𝒫S simulates an interaction between the composed protocol and 𝒫T. 𝒫S decides which queries it will submit in its interaction with multiple instances of the protocol that claims to solve 𝒮, based on the queries it sees in this simulation. We will show that 𝒫S wins in the interaction since 𝒫T wins in its interaction with 𝒯 by assumption.

2 Preliminaries

2.1 NIIS model

A snapshot object consists of an array of elements and supports two types of operation: write and read. In a write operation, a process with id i writes a value to the i-th cell of the array. In a snapshot operation, a process atomically reads the contents of all the cells of the array. Similarly, the immediate snapshot (IS) object was introduced by Borowsky and Gafni in [8]. An IS object consists of an array of elements and provides writeread operations, where a process with id i writes a value to the i-th element of the array and returns a snapshot of the array immediately after this write. The writeread operations on a single IS object by multiple processes are said to be concurrent if all their snapshot operations happen after all their writes to the array are finished.

IS objects are the basic building blocks of the iterated immediate snapshot (IIS) model. The IIS model consists of n processes Π={u1,u2un} and a bounded sequence of IS objects {IS1,IS2ISe}. In the execution of a protocol in the IIS model, each process will terminate after performing a writeread operation on each IS object in ascending order in the sequence. The IIS model has been shown to have computational power equivalent to that of the standard shared memory model in [1, 8].

Hoest and Shavit [15] introduced a new model called the non-uniform iterated snapshot (NIIS) model, a variant of the iterated immediate snapshot (IIS) model. The NIIS model consists of n processes Π={u1,u2un} and an unbounded sequence of IS objects {IS1,IS2}. Each process starts with a process id i and its input value vi, and performs a writeread operation on each IS object sequentially. By definition, each NIIS protocol is determined by a decision map δ from a local state to output values (or a special value ). In the execution of an NIIS protocol δ, each time a process completes a writeread operation, it checks whether it has reached a final state by applying δ to its local state variable. If δ returns , the process continues to access the next IS object. Otherwise, the process uses the return value of δ as its output and terminates. An NIIS protocol is said to be transparent to some entity if this entity knows everything about the function δ. In this paper, we consider full-information NIIS protocols in which each process uses its current state as an argument to perform a writeread operation. After a writeread operation, its new state consists of its process id i and the result of the writeread operation. Note that unlike the IIS model, the NIIS model allows different processes to terminate after accessing a different number of IS objects.

A configuration of a protocol consists of the state of each process and the contents of each IS object. Since each process remembers its entire history and only process pi can write to the i-th component of each IS object, the state of each IS object is included in states of the processes, which means that a configuration is fully determined by the states of processes. A process is active in a configuration if it is not terminated. A configuration is final if it has no active processes. If C is a configuration and U is a set of processes that are poised to perform writeread operations on the same IS object, then C{U} is the configuration that results from C when the processes in U concurrently take a step in the protocol. A schedule of a protocol from C0 is a finite or infinite sequence of sets of processes γ=U1,U2 such that there is a sequence of configurations C0,C1 where processes in Ui is active in Ci1 and Ci=Ci1Ui for all Ui in γ. A U-only schedule from C is a schedule in which only processes in U appear. Each finite schedule from an initial configuration results in a reachable configuration. A protocol is wait-free if each process is terminated after a finite number of steps.

A task 𝒯 consists of a finite set of input vectors, I, a finite set of output vectors, O, and a map T:I2O that represents the task specification. For each input vector xI, T maps x to a non-empty set T(x)O of output vectors. Suppose that A is a wait-free protocol that has an initial configuration with input vector x for each xI. We say that a protocol A solves 𝒯 if, for each xI, every final configuration that is reached from the initial configuration of A that has input vector x has output vector y and yT(x). If 𝒯 and 𝒯 are tasks with maps T and T, respectively, their composition 𝒯𝒯 is a task that has the same set of input vectors as 𝒯 and the same set of output vectors as 𝒯. The map of the composed protocol is defined by TT(x)={T(y)|yT(x)}.

2.2 Extension-based proofs

In [3], Alistarh, Aspnes, Ellen, Gelashvili and Zhu formally defined the class of extension-based proofs that generalizes the impossibility proof technique used in the FLP impossibility result. The proof of impossibility is modeled as an interaction between a prover and a wait-free NIIS protocol that claims to solve the task. The protocol is defined by a map δ from the process states to the output values or a special value . The prover asks queries to learn information about the protocol, such as the δ values of states of processes in some reached configuration, in an effort to find a violation of the task specification, or construct an infinite execution.

If there exists a prover that can defeat any protocol that claims to solve the task, we say that the task has an extension-based impossibility proof. But if for each prover, an adversary can adaptively design a protocol based on the queries made by the prover such that the prover cannot win in the interaction, there is no extension-based proof for the impossibility of the task. The adaptive protocol constructed by the adversary will be referred to as an adversarial protocol as in [3].

Three classes of extension-based proofs are defined in [3, 10]. We will first introduce restricted extension-based proofs. The interaction between the prover and a protocol proceeds in phases. In each phase φ the prover starts with a finite schedule α(φ) and a set of configurations 𝒜(φ) reached from some initial configurations of the task by α(φ) which only differ in the input values of processes that are not in the schedule α. For φ=1, the schedule α(1) is the empty schedule and 𝒜(1) contains all initial configurations. The set of configurations reached in phase φ is denoted by 𝒜(φ) which is empty at the beginning of phase φ. The prover queries the protocol by choosing a configuration C𝒜(φ)𝒜(φ) and a set of processes U that are poised to perform writeread operations on the same IS object in C. The protocol replies with the value of δ of each process in U in the configuration C resulting from the processes in U concurrently performing writeread operations, and the prover adds the configuration C to 𝒜(φ). A chain of queries is a (finite or infinite) sequence of queries such that if (Ci,Ui) and (Ci+1,Ui+1) are consecutive queries in the chain, then Ci+1 is the configuration resulting from scheduling Ui from Ci. The prover is allowed to construct finitely many chains of queries in a phase.

When the output values from the response of a query are not allowed output values for the task, we say that the prover finds a violation of the task specification. Similarly, if a prover can construct an infinite chain of queries, the protocol must admit that it is not wait-free. If the prover does not find a violation or construct an infinite execution after a finite number of queries or chains of queries, it must end the current phase and choose some configuration C𝒜(φ). Let α be the schedule such that C is reached via α from some configuration in 𝒜(φ). The schedule α(φ)α will be used as the initial schedule α(φ+1) in the next phase. Suppose that the configuration C is reached from an initial configuration Cini. Then 𝒜(φ+1) consists of all configurations reached by the schedule α(φ+1) from initial configurations that only differ from Cini by the input values of the processes that do not appear in α(φ+1).

Extension-based proofs allow for an additional type of query. The prover performs an output query by choosing a configuration C𝒜(φ)𝒜(φ), a set of processes U that are poised to access the same IS object, and a value y{0,1,,k}. If there is a U-only schedule from C that results in a configuration in which a process in U outputs y, the protocol will return some such schedule. Otherwise, the protocol returns NULL. Augmented extension-based proofs provide a more general query, called an assignment query. An assignment query consists of a configuration C𝒜(φ)𝒜(φ), a set of processes U, and an assignment function f from a subset U of U to the output values. If there is a U-only schedule from C that results in a configuration in which the output value of each process u in U is f(u), the protocol will return some such schedule. Otherwise, the protocol will return NULL. It is shown that an output query (C,U,y) can be simulated by a sequence of assignment queries (C,U,fu) for each process u in [10]. Therefore, we will discuss only augmented extension-based proofs in subsequent sections.

The tasks which have been proved to have no extension-based proofs of the impossibility are still quite limited. It is proved in [3, 2] that the (n,k)-set agreement task does not have extension-based proofs of the impossibility in the NIIS and NIS models. Some similar results on the approximate agreement task are proved in [4, 16]. Shi and Liu [18] gave a topological characterization of a colorless task to have no extension-based proofs of impossibility.

Attiya, Castañeda, and Rajsbaum [5] studied the limitations of valency arguments in the IIS model. Note that in the IS and IIS models, extension-based proofs are too powerful. They define the class of local valency impossibility proofs, that cannot show there is no wait-free protocol for (n,n1)-set agreement, weak symmetry breaking or (2n2)-renaming in the IIS model. Attiya, Fraigniaud, Paz and Rajsbaum [6] introduced an FLP-style impossibility proof which is a simple case of extension-based proofs and local valency impossibility proofs, and showed that there are no FLP-style proofs of the impossibility of any 1-dimensional colorless task if it is not wait-free solvable.

2.3 A class of reductions in the NIIS model

If a task 𝒯 reduces to a task 𝒮, then there exists a protocol Al+1AlAlA1A1 that solves the task 𝒯 where each Ai is a protocol to solve the task 𝒮 and each Ai is an NIIS protocol for some task i. We assume that the number of instances of the protocol to solve 𝒮 is a constant. We say that (1,2,.l,l+1) is a reduction from a task 𝒯 to a task 𝒮.

Previous work[10] used a restricted version of this definition in which only one instance in this composition is a protocol that solves the task 𝒮. The restricted version of reductions in their paper is enough to prove the non-existence of extension-based proofs for many tasks. But some reductions use multiple instances of 𝒮, such as the reduction given by [13] from weak symmetry breaking to (n1)-set agreement, which uses two instances of (n1)-set agreement.

3 Composing NIIS protocols

Let (1,2,.l,l+1) be a reduction from a task 𝒯 to a task 𝒮. Suppose that some protocol, denoted as A, claims that it solve the task 𝒯. We denote the NIIS protocols that solve 1,2,l,l+1 by A1,Al,Al+1. A difference between A and Ai, for each 1il+1, is that everything about Ai is known, which is not true for A. We can assume without loss of generality that all processes will terminate after accessing the same number of IS objects in the execution of Ai. The composed protocol Al+1AlAlA1A1 should solve 𝒯 where each Ai is an instance of A. Note that each Ai is an NIIS protocol, and we cannot assume that all processes will terminate after accessing the same number of IS objects in the execution of Ai.

We consider the case where there are two NIIS protocols, since a more general composition can be easily obtained if we can compose two NIIS protocols.

3.1 Composing two NIIS protocols

The composition of NIIS protocols is much more complicated than the composition of IIS protocols, as processes can terminate after different rounds during the execution of some NIIS protocol. To compose IIS protocols, we can simply have each process execute IIS protocols sequentially. Since each process ends the execution of a protocol after the same round, the information used in the execution of different IIS protocols will not be mixed together. But this is not true for NIIS protocols, since processes accessing the same IS object in the composed protocol may be running different NIIS protocols. We need a mechanism to separate information from different NIIS protocols.

Let A1 and A2 be NIIS protocols such that each output from A1 is a possible input for A2. We temporarily assume that everything about A1 and A2 is known (this is not true in the construction of 𝒫S). In [10], Brusse and Ellen provided an implementation of the composed protocol A2A1 in the NIS model using estimate vectors introduced by Borowsky and Gafni [9]. The composition of two NIIS protocols in our paper can use almost identical techniques. But since this is quite important in this paper, we have to explain it in detail. Let ISh, ISh′′ and ISh denote the h-th IS object accessed by processes in the schedules of A1, A2, and A2A1, respectively. Each process ui simulates the steps of ui in protocol A1 followed by the steps of ui in protocol A2. The result of a writeread operation in simulation of A1 can be obtained from some writeread operation of the composed protocol. Since different processes may terminate after different rounds in A1, the result of a writeread operation in the simulation of A2 cannot be easily obtained. Each process locally maintains an infinite sequence of estimate vectors, each with n components. The h-th estimate vector of process ui is a potential output of the writeread operation on ISh′′ in the simulation of A2.

Let γ be a schedule of the composed protocol A2A1 from some initial configuration Cini. Each process updates its estimate vectors during this schedule of the composed protocol A2A1, and finalizes its i-th estimate vector after collecting enough information. This estimate vector can be proved to be equivalent to the return value of the writeread operation to ISi′′ by the same process in some schedule of the second NIIS protocol. We can call this schedule the restriction of γ to A2. A process will apply the function δ that specifies the second protocol to the finalized estimate vector to decide whether it should end the simulation of A2 (and which value to output if so).

We give some pseudocode in Algorithm 1. The process ui performs writeread operations to IS1,IS2 sequentially, with hi indicating the next IS object to be accessed. The value written to ISi by a process consists of three parts: r (round number), state1 (state in the simulation of A1) and state2 (state in the simulation of A2). The round number is 0 when the process is simulating A1, is r when simulating the r-round of A2. The attribute state2 is a sequence of estimate vectors.

When a process simulates A1, it computes the result of the writeread operation on ISi from the result of the writeread operation on ISi by ignoring the attribute state2. However, a process ui will update its estimate vectors using the result as described in Algorithm 2. ui updates the k-th component of its k-th estimate vector when this component is blank and the result of a writeread operation on IShi contains some estimate vectors from some other process, where the k-th component of its k-th estimate vector is not blank. After a process ui has finished simulating A1, it modifies its first estimate vector to include its output of A1 as the input for A2 and then tries to finalize it. To finalize its ri-th estimate vector, for any ri1, the process repeatedly performs a writeread operation on the next IS object IShi (to contain its first ri estimate vectors). The process updates its estimate vectors just as it does when simulating A1. The process does not finalize its ri-th estimate vector in two cases. The first case is that it sees some other process ui accessing IShi when that process was simulating A1 or an earlier round of A2. This is to avoid process ui, which is simulating an earlier round, from missing the finalized ri-th estimate vector of process ui. The other case is that it changes its ri-th estimate vector using information from IShi.

Otherwise, the process finalizes its ri-th estimate vector. When a process finalizes its ri-th estimate vector, this vector will be used as the output of its writeread operation on ISri′′ in the simulation of A2. Note that it may take many rounds of the composed protocol for a process to finalize its current estimate vector, i.e. to take a step in the simulated execution. Based on the return values of its ri-th estimate vector, the process terminates its simulation of A2 or modifies its (ri+1)-th estimate vector to include its new state in the simulation of A2 and continues to finalize its (ri+1)-th estimate vector.

Algorithm 1 The composition of two NIIS protocols A1 and A2.
Algorithm 2 update estimate vectors.

3.2 Some properties of the composition

Now we present some properties that are necessary in Section 4. The first property was shown in [10].

Lemma 1.

The composed protocol A2A1 is wait-free if both A1 and A2 are wait-free.

The second property is about restrictions. Given an initial configuration Cini of A2A1, the composed protocol simulates a schedule of A1 from Cini, followed by a schedule of A2, where the output of the first schedule serves as input to the second schedule. Let γ be a schedule of A2A1 from Cini. The restriction of γ to A1 is defined as the schedule γ1 obtained from γ by removing all the occurrences of each process after simulating A1. Each process ui finishes its simulation of A1 with output yi during the schedule γ of A2A1 from Cini if and only if ui terminates with output yi during the schedule γ1 of A1 from Cini.

Let U be the set of processes that complete their simulation of A1 during γ and, for each uiU, let yi be its simulated output from A1. Let C2 be any initial configuration of A2 in which each process uiU has input yi. We have to prove that, for each l, the l-th estimate vectors that have been finalized by the processes in U correspond to the responses of the writeread operations in some U-only schedule γ2 of A2 from C2. Each process ui finishes its simulation of A2 with output zi during the schedule γ of A2A1 from Cini if and only if ui terminates with output zi during the schedule γ2 of A2 from C2. The restriction of γ to A2 is defined as the schedule γ2. Note that we are using the same definition as in [10] except that we are using the NIIS model rather than the NIS model.

Lemma 2.

The estimate vectors finalized by the processes during the schedule γ of A2A1 from Cini correspond to the responses of the writeread operations during the schedule γ2, which is the restriction of γ to A2, starting from C2.

Proof.

For any simulated IS object ISr′′ of A2, let Qh denote the set of processes that have finalized their r-th estimate vectors after performing writeread operations to ISh of the composed protocol A2A1. For any indices j, j and k, if Ej[r][k] and Ej[r][k], then Ej[r][k]=Ej[r][k]. This is because the value Ej[r][k] can only be defined on line 40 by the process with id k, and other processes are only allowed to copy this value into their estimate vectors on line 9 of Algorithm 2. We can define a partial order on the r-th estimate vectors. Given the indices j and j, if for any k, Ej[r][k]= or Ej[r][k]=Ej[r][k], then Ej[r] is ordered before Ej[r] (i.e. Ej[r] contains each non-blank value in Ej[r]). If there is a partial order defined on the set of finalized r-th estimate vectors, then these finalized r-th estimate vectors can be seen as the result of writeread operations on ISr′′.

Suppose that the process ui is in Qh+1 but not in Qh, i.e. ui finalizes its r-th estimate vector after accessing Qh. Consider the response of the writeread operation by a process ui on ISh. According to our implementation (checks on can_finalize in Algorithm 1), the r-th estimate vector of the process ui is ordered after or concurrently with the r-th estimate vector of ui if states_of_processes[i] (Line 25 in algorithm 1). For other index i where states_of_processes[i]=, the process ui will see the r-th finalized estimate vector of ui before finalizing its r-th estimate vector, since ui always includes Ei[r] in its written value to ISh+1,ISh+2. This means that the r-th finalized estimate vector of ui is ordered before that of ui. There exists a total order on the finalized r-th estimate vectors.

For each index r, we divide the processes that have finalized its r-th estimate vector into a sequence SEQr of sets, where processes having the same r-th estimate vector are in the same set. These sets are ordered using the partial order on the finalized r-th estimate vectors. The restriction of γ to A2 is defined as the concatenation SEQ1SEQ2.

The last property is related to extension-based proofs. An assumption of the algorithm to compose two NIIS protocols is that the two NIIS protocols are transparent i.e. the entity that tries to compose them knows everything about the functions δ1,δ2. In the definition of extension-based proofs, the protocol that claims to solve 𝒮 is not transparent. We show that if an entity is allowed to interact with NIIS protocols that are not transparent just as a prover does, this entity can still compose two NIIS protocols.

Lemma 3.

When two NIIS protocols are not transparent protocols, if an entity are allowed to submit queries to the protocols that are not transparent, the information obtained through queries is sufficient for the composition of NIIS protocols.

Proof.

When composing two NIIS protocols A1A2, updates and finalization of estimate vectors are independent of the protocols A1,A2. The functions δ1,δ2 representing protocols A1,A2 are only used when some estimate vector is finalized to determine whether to terminate the simulation of A1,A2 on line 7, 19 of Algorithm 1. By assumption, this entity can submit queries to the protocol to learn about this information.

3.3 Composing multiple NIIS protocols

The composition of multiple NIIS protocols AmA2A1 can be derived from the composition of two NIIS protocols. Inductively, AmA2A1 is the composition of A1 and AmA3A2. Let γ be a schedule of AmA2A1 from some initial configuration C. Let γ(1) denote the restriction of γ to A1, γ(2) denote the restriction of γ to AmA3A2. The restriction of γ to Aj, where 2jm, can be defined inductively as the restriction of γ(2) to Aj. The properties proved in Section 3.2 also apply to the composition of multiple NIIS protocols.

4 Reductions and extension-based proofs

4.1 Simulation

In the following sections, we extend the proof in [10] to more general reductions. Suppose that 𝒯 reduces to 𝒮 and 𝒫T is an augmented extension-based prover that can win against any protocol that claims to solve 𝒯. Our goal is to construct an augmented extension-based prover 𝒫S for task 𝒮 that can also win against any protocol A that claims to solve S.

𝒫S will simulate an interaction between 𝒫T and the composed protocol Al+1AlAlA1A1, and will use restrictions to interact with multiple copies of A, as shown in Figure 1. There is an interaction between 𝒫T and the composed protocol in Figure 1. But at the same time, if we regard the elements within the black frame as a prover 𝒫S, the interaction can be seen to happen between 𝒫S and multiple instances of A.

If the prover 𝒫T submits a query in the simulation, the prover 𝒫S will reinterpret this query as queries to some instances of A and will use the responses of the instances of the protocol A and its knowledge about the protocols A1,Al,Al+1 to generate a response to the original query. The most important idea of this section is that during the interaction between 𝒫T and the composed protocol, 𝒫S interacts with each instance Ai to resume the logical executions of the NIIS protocols. Since the prover 𝒫T is going to win against the composed protocol, there must be something wrong within the workflow. This can only happen in interactions between 𝒫S and some copy of A, which means that 𝒫S wins in the interaction with some copy.

Refer to caption
Figure 1: Simulation workflow.

4.2 Multiple-instance extension-based proofs

The prover 𝒫S will independently interact with a constant number of instances of the protocol A. Note that we are using a different definition of prover here. In the standard definition, an extension-based prover interacts with a single instance of the protocol. A multiple-instance extension-based prover interacts with multiple instances of the same protocol A and maintains a separate set of arguments (i.e. all arguments an extension-based prover will maintain during its interaction with a protocol) recording its interaction with each instance of A. For example, ϕi and ϕj, representing the current phases of interactions with Ai and Aj can be different.

A multiple-instances prover submits queries to each instance individually. It can use the response from any query to decide what to do next (such as how to submit a query in the interaction with another instance of A). But we do not allow a multiple-instance prover to use responses from different instances to produce a conflict even when the protocol A responds with different output values for a single schedule in its interaction with different instances. A multiple-instance prover wins if it can win in the interaction with some instance using only the information obtained from this instance. Otherwise, the protocol wins. It is easy to see that a multiple-instance extension-based prover is at least as powerful as an extension-based prover since a multiple-instance extension-based prover can choose to interact with one instance only.

4.3 Construction of 𝓟𝑺

Let γ be a schedule of the composed protocol Al+1AlAlA1A1 from some initial configuration Cini to some configuration C. The restriction of γ to a protocol Ai or Ai is defined in Section 3.3, denoted by γi and γi. By Lemma 3, given a schedule γ of the composed protocol, the prover 𝒫S can compute the restriction of γ to any Ai or Ai.

An initial configuration Ci of Ai is defined as compatible with some configuration Ciγi of Ai if the output value of every process terminated in Ciγi is the input value of the same process in Ci. An initial configuration Ci+1 of Ai+1 is defined to be compatible with some configuration Ciγi of Ai if the output value of every process terminated in Ciγi is the input value of the same process in Ci+1.

In the simulation, 𝒫S maintains the following invariant. Suppose that 𝒫T has reached the configuration C=Ciniγ of the composed protocol Al+1AlAlA1A1. Then there exist initial configurations C1,C1,C2,,Cl,Cl+1 of the protocols A1,A1,A2,Al,Al+1 such that, for all 1il, Ci is compatible with Ciγi, Ci+1 is compatible with Ciγi, and 𝒫S has reached configuration Ciγi in its interaction with Ai. Let ψ denote the current phase of the simulated interaction, and let ϕi denote the current phase of Ai for each 1il. αi(ϕi) is a prefix of γi for each 1il.

At the beginning of the simulation, 𝒫T starts with an empty schedule and has reached the initial configurations of the composed protocol Al+1AlAlA1A1. For each 1il, the interaction between 𝒫S and Ai also starts with an empty schedule and has reached the initial configurations of Ai. Since every initial configuration of Ai is compatible with every initial configuration of Ai, the invariant holds at the start of phase 1 of the simulated interaction. We will discuss the strategy that the prover 𝒫S will use in interactions with these instances of A when 𝒫T submits a query, submits an assignment query, or ends a phase in the simulation.

4.3.1 Responding to queries

Suppose that the prover 𝒫T submits a query (C,U) in phase ψ of the simulated interaction, where C is some configuration and U is a set of processes that are poised to access the same IS object. The prover 𝒫S must respond with the δ values of the processes in U in the configuration C{U} resulting from scheduling U from C in the composed protocol. We can express the configuration C as Ciniα(ψ)β where Cini is an initial configuration since the configuration C has been reached in phase ψ. Let γ=α(ψ)β. The invariant holds before the query: Each interaction instance has reached some configuration Ciγi where Ci is an initial configuration of Ai compatible with Ciγi. And Ci+1 is compatible with Ciγi.

The processes in U in the configuration C of the composed protocol Al+1AlAlA1A1 can be simulating different protocols as described in Section 3.1. For each process u in U, the NIIS protocol that u is simulating can be calculated. Therefore, we can separate the processes in U into groups {U1,U1,Ul,Ul,Ul+1}. For example, U1 consists of the processes that simulate A1. If some process u in some group Uj or Uj finalizes its current estimate vector (or is simulating the first protocol A1) after accessing the new IS object, u takes a step in its current simulated NIIS protocol. Otherwise, it will not take a step in its current simulation. Let Vi and Vi be the set of processes that take a step in the simulated protocol Ai and Ai, respectively. It is easy to see that ViUi and ViUi. Let Vi,k be the processes in Vi that are poised to access the k-th simulated IS object. And with the same definition, we have Vi,k. There are several cases, depending on the steps that the processes in Vi,k or Vi,k take in the simulation.

If processes in Vi,k simulate some Ai and do not execute the last step, they still simulate Ai after taking a step. Let Cres be the configuration resulting from scheduling Vi,k from Ciγi. No more processes terminate in the simulated execution of Ai, and Ci is compatible with Cres. The invariant continues to hold. 𝒫S returns the states of Vi,k in Cres to 𝒫T.

If processes in Vi,k simulate some Ai and execute the last step, each process will output a value and terminate in the simulation of Ai since processes terminate after accessing the same number of IS objects in any execution of Ai (argued in the beginning of Section 3). Let Cres be the configuration resulting from scheduling Vi,k from Ciγi. 𝒫S returns the states of Vi,k in Cres to 𝒫T. Ci has to be changed since more processes have been terminated in the simulation of Ai. We can choose a new initial configuration Ci of Ai in which each newly terminated process p will use the output of Ai as input and every other process has the same input as the original Ci. The new initial configuration Ci is compatible with Cres. The newly terminated processes do not appear in γi, so γi is a valid schedule from the new initial configuration Ci. Invariant holds in this case.

Suppose that the processes in Vi,k simulate some Ai. The prover 𝒫S can submit the query (Ciγi,Vi,k) to the protocol Ai and return a response to 𝒫T. If no process terminates in the simulation, the invariant holds, since γi{Vi,k} is the new restriction to Ai and will not change the initial values of active processes in the simulation of Ai or Ai+1. Otherwise, some processes in Vi,k terminate in the simulation of Ai, and the initial configuration Ci+1 of Ai+1 must be changed. For each terminated process u in Vi,k, the input of Ai+1 is the output of Ai. Let Cres be the configuration resulting from scheduling Vi,k from Ciγi. The new initial configuration Ci+1 of Ai+1 is compatible with Cres. The invariant still holds.

The prover 𝒫S may submit multiple queries (Ciγi,Vi,k) for different k to some protocol Ai. Operations on different simulated IS objects will not affect each other. Therefore, submitting these queries in different orders will not change the response of each query and the internal variables (such as the set of configurations reached in phase ϕi) of the interaction between 𝒫S and Ai. Therefore, the invariant holds after all these queries.

4.3.2 Responding to assignment queries

Suppose that the prover 𝒫T submits an assignment query (C,U,f) in phase ψ of the simulated interaction, where U is a set of processes and f is a function from a subset U of U to the output values. The prover 𝒫S has to reply to 𝒫T whether there exists a U-only schedule from C to a configuration in which the output value of each process u in U is f(u). We will say that this resulting configuration satisfies the requirement of f. Since 𝒫T will not reach any new configuration to respond to assignment queries, the invariant will continue to hold.

Let γ be the schedule from some initial configuration to C. By the invariant, for 1il, 𝒫S has reached the configuration Ciγi in the i-th interaction with A, for some initial configuration Ci of protocol Ai compatible with Ciγi, where γi and γi are restrictions to Ai and Ai. And Ci+1 is compatible with Ciγi.

The prover 𝒫S will treat all protocols Ai and Ai as black boxes that allow assignment queries. This is possible for each Ai since the prover 𝒫S can submit assignment queries to protocol Ai. The prover 𝒫S knows everything about the protocol Ai, so assignment queries are also possible for Ai. We regard protocols of both types as black boxes to simplify our proof. At first, the prover 𝒫S computes a set of assignment queries to AlAlA1A1 (note that Al+1 does not appear), denoted by F2l. The prover 𝒫S repeats this computation by removing the last black box until only one black box remains. We define F2l+1 as the original query (C,U,f).

First, 𝒫S computes F2l, which is the set of assignment queries to AlAlA1A1. In the following discussions, we emphasize that schedules for different protocols are used. Whenever we talk about a schedule, we explicitly mention the protocol. Let C2l+1c be some initial configuration of the protocol Al+1 compatible with Clγl, where there is a U-only schedule β of Al+1 from C2l+1cγl+1 to a configuration that satisfies the requirement of f. Let U denote the set of processes obtained from U by removing the processes that appear in γl+1.

Let C be the configuration of the composed protocol AlAlA1A1 (note that Al+1 is removed here), where the processes in Π(UU) are in the same state as the configuration C and the processes in UU have terminated with their input values in Cl+1. Intuitively, suppose that the configuration C is reached from some initial configuration Cini by a schedule γ, then C is also reached from Cini by γ, where terminated processes omit the additional steps assigned by the schedule. Now we will prove that there exists a U-only schedule of the composed protocol AlAlA1A1 from C to some configuration where the output values of U are the input values of C2l+1c if and only if there exists a U-only schedule of Al+1AlAlA1A1 from C to some configuration whose output values satisfy f.

Suppose that there is a U-only schedule β of AlA1 from C to a configuration whose output values are the input values of C2l+1c. β can be used directly as a schedule of Al+1AlA1. We cannot concatenate the schedule β and β, since β (a schedule of the protocol Al+1) cannot be used directly as a schedule of Al+1AlA1. However, we can construct a U-only schedule βc of Al+1AlAlA1A1 whose restriction to Al+1 is γl+1β. Let β={U1}{U2}{Ue}, then βc is defined as β{U1}{U1}{U2}{U2}{Ue}{Ue} such that for each set of processes Ui in β, processes in Ui concurrently access IS objects until they all finalize their i-th estimate vectors when they are poised to access the i-th IS object in the simulation of Al+1. Their estimate vectors are the same according to the checks performed when an estimate vector is finalized. ββc is a U-only schedule from C to a configuration that satisfies the requirement of the function f.

If there is a U-only schedule β′′ from C to some configuration Cd that satisfies the requirement of f, Cd can be seen as an output configuration of the last protocol Al+1. Therefore, there exists an initial configuration C2l+1c of the protocol Al+1 compatible with Clγl, and there is a U-only schedule β of Al+1 from Cl+1γl+1 to Cd. Consider the U-only schedule β′′=c1c2cm from C to Cd of the composed protocol, where each ci is a set of processes. We will construct a U-only schedule based on β′′ by stopping the processes that simulate Al+1 from taking actions. Let E=e1e2em be a sequence, where ei is the set of processes that simulate Al+1 in the configuration reached from C by the schedule c1c2ci. The processes in UU already simulate the protocol Al+1, so UUe1e2em. The schedule c1e1,c2e2cmem is a U-only schedule of the composed protocol. Deleting the processes in ei will not change the estimate vectors finalized by processes that do not simulate Al+1, since the processes in ei write only information about the simulation of Al+1, which will not affect the simulation of previous NIIS protocols. Therefore, c1e1,c2e2cmem is a U-only schedule from C to a configuration where the output values are the input values of C2l+1c.

For each such initial configuration C2l+1c, we can submit an assignment query (C,U,f) to the composed protocol AlAlA1A1 in which f is defined as a function from the processes in U to their output values in configuration C2l+1c. If a U-only schedule is returned, there is a U-only schedule for the assignment query (C,U,f) to Al+1AlAlA1A1. Otherwise, the prover 𝒫T receives the response NULL. F2l consists of all these assignment queries (C,U,f).

We continue to trace back: Suppose that Fi+1 is already calculated. For each assignment query (C,U,f) in Fi+1, the prover 𝒫S computes each initial configuration Ci+1c. Ci+1c has two requirements. It must be compatible with Ceγe where Ce is the initial configuration (some Cj or Cj in the invariant) of the i-th black box and γe is the restriction (some γj or γj in the invariant) to this black box, and there must exist a U-only schedule β of the (i+1)-th black box from Ci+1cγe+1 to a configuration that satisfies the function f, where γe+1 is the restriction of γ to the (i+1)-th black box. To decide whether such a U-only schedule β exists, the prover 𝒫S can submit assignment queries to the (i+1)-th black box. Let U′′ denote the set of processes obtained from U by removing the processes in γe+1. For each initial configuration Ci+1c, there is an assignment query (C′′,U′′,f′′) in which f′′ is defined as a function from the processes in U′′ to the values in the configuration Ci+1c. As proved in the above example, there is a U′′-only schedule from C′′ to a configuration whose output values are the input values of Ci+1c if and only if there is a U-only schedule that can serve as a response to the assignment query (C,U,f). Fi consists of all these assignment queries (C′′,U′′,f′′).

This procedure ends when F0 is computed. If F0 is not empty, there exists a U-only schedule from C to some configuration that satisfies the requirement of f. Otherwise, the prover 𝒫T gets a response of NULL.

4.3.3 Ending phases

Suppose that the prover 𝒫T decides to end its current phase ψ by choosing a configuration C=Ciniγ of the composed protocol Al+1AlAlA1A1. By the invariant, 𝒫S has reached configuration Ciγi in the i-th interaction with protocol A, for some initial configuration Ci of Ai.

For Ai that does not have an interaction with 𝒫S during the phase ψ of the simulation, the prover 𝒫S does nothing. For a protocol instance Ai in which at least one process has taken a simulation step during the schedule γ of the composed protocol, the prover 𝒫T has reached the configuration Ciγi. The prover 𝒫S ends phase ϕi by choosing configuration Ciγi, sets αi(ϕi+1)=γi and proceeds to the next phase in the interaction with Ai.

4.3.4 The prover 𝓟𝑺 will win

The prover 𝒫T will finally win in the simulated interaction with the composed protocol Al+1AlAlA1A1. We will analyze all possible circumstances and show that 𝒫S will win against one of the instances of the protocol A.

The first case is that the prover 𝒫T finds a configuration, C, in which the output values violate the specification of the task 𝒯. For example, the processes in C output different values when 𝒯 is the consensus task. By the invariant, the prover 𝒫S has reached configuration Ciγi in the interaction with Ai. Suppose that for each interaction instance, the output values of the processes in Ciγi satisfy the task specification of 𝒮. The output values in configuration C will satisfy the task specification of 𝒯 as 𝒯 reduces to 𝒮, which is a contradiction. Therefore, there exists an interaction with some Ai that reaches a configuration with incorrect output values. The prover 𝒫S finds a violation of the task specification and wins.

The second case is that the prover 𝒫T constructs an infinite chain of queries in the interaction. Each protocol Ai is wait-free, so an infinite schedule is not possible in it. Estimation vectors will not cause an infinite schedule of the composed protocol, as proved in [10]. Therefore, there must be an interaction with some A instance in which the prover 𝒫S also constructs an infinite chain of queries. In this situation, the prover 𝒫S wins.

The last case is that the prover 𝒫T constructs infinite phases in the simulation. By definition, 𝒫T can submit finitely many queries or assignment queries in one phase. Each time the prover 𝒫T ends its phase by choosing a schedule, at least one step is taken by some set of processes. Since each protocol Ai is wait-free, the prover 𝒫T can only end a finite number of phases by simulating only these protocols. For other phases, there must be some protocol Ai that will make progress and, therefore, end its current phase after the prover 𝒫T ends a finite number of phases (because an estimate vector can take several rounds to finalize). The number of A protocol instances is finite, so there must be an interaction with some A instance in which the prover 𝒫S also constructs infinite phases. The prover 𝒫S still wins.

Theorem 4.

If a task 𝒯 reduces to a task 𝒮 using multiple instances of task 𝒮 and there is an augmented extension-based proof of the impossibility of solving the task 𝒯 in a wait-free manner in the NIIS model, then there is an multiple-instance extension-based proof of the impossibility of solving the task 𝒮 in a wait-free manner in the NIIS model.

5 Conclusions

In this paper, we partially solve an open question proposed in [10]. If there is a reduction using multiple instances of 𝒮 from 𝒯 to 𝒮 and there is no multiple-instance extension-based proof for 𝒮, then there is no augmented extension-based proof for 𝒯. Although many reductions that appear in the literature reduce to a single instance of a problem, our result applies to reductions that reduce to a constant number of different instances of that problem. An open problem that remains is to extend our results to more general reductions (such as Turing reductions). Another open problem is whether multiple-instance extension-based provers are strictly more powerful than extension-based provers.

References

  • [1] Yehuda Afek, Hagit Attiya, Danny Dolev, Eli Gafni, Michael Merritt, and Nir Shavit. Atomic snapshots of shared memory. J. ACM, 40(4):873–890, September 1993. doi:10.1145/153724.153741.
  • [2] Dan Alistarh, James Aspnes, Faith Ellen, Rati Gelashvili, and Leqi Zhu. Why extension-based proofs fail. Proceedings of the 51’st Annual ACM Symposium on Theory of Computing (STOC), pages 986–996, 2019. doi:10.1145/3313276.3316407.
  • [3] Dan Alistarh, James Aspnes, Faith Ellen, Rati Gelashvili, and Leqi Zhu. Why extension-based proofs fail. SIAM Journal on Computing, 52(4):913–944, 2023. doi:10.1137/20M1375851.
  • [4] Dan Alistarh, Faith Ellen, and Joel Rybicki. Wait-free approximate agreement on graphs. In Structural Information and Communication Complexity: 28th International Colloquium, SIROCCO 2021, Wrocław, Poland, June 28 – July 1, 2021, Proceedings, pages 87–105, Berlin, Heidelberg, 2021. Springer-Verlag. doi:10.1007/978-3-030-79527-6_6.
  • [5] Hagit Attiya, Armando Castañeda, and Sergio Rajsbaum. Locally solvable tasks and the limitations of valency arguments. Journal of Parallel and Distributed Computing, 176:28–40, 2023. doi:10.1016/j.jpdc.2023.02.002.
  • [6] Hagit Attiya, Pierre Fraigniaud, Ami Paz, and Sergio Rajsbaum. One Step Forward, One Step Back: FLP-Style Proofs and the Round-Reduction Technique for Colorless Tasks. In Rotem Oshman, editor, 37th International Symposium on Distributed Computing (DISC 2023), volume 281 of Leibniz International Proceedings in Informatics (LIPIcs), pages 4:1–4:23, Dagstuhl, Germany, 2023. Schloss Dagstuhl – Leibniz-Zentrum für Informatik. doi:10.4230/LIPIcs.DISC.2023.4.
  • [7] Elizabeth Borowsky and Eli Gafni. Generalized flp impossibility result for t-resilient asynchronous computations. In Proceedings of the Twenty-Fifth Annual ACM Symposium on Theory of Computing, STOC ’93, pages 91–100, New York, NY, USA, 1993. Association for Computing Machinery. doi:10.1145/167088.167119.
  • [8] Elizabeth Borowsky and Eli Gafni. Immediate atomic snapshots and fast renaming. In Proceedings of the Twelfth Annual ACM Symposium on Principles of Distributed Computing, PODC ’93, pages 41–51, New York, NY, USA, 1993. Association for Computing Machinery. doi:10.1145/164051.164056.
  • [9] Elizabeth Borowsky and Eli Gafni. A simple algorithmically reasoned characterization of wait-free computation (extended abstract). In Proceedings of the Sixteenth Annual ACM Symposium on Principles of Distributed Computing, PODC ’97, pages 189–198, New York, NY, USA, 1997. Association for Computing Machinery. doi:10.1145/259380.259439.
  • [10] Kayman Brusse and Faith Ellen. Reductions and extension-based proofs. In Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing, PODC’21, pages 497–507, New York, NY, USA, 2021. Association for Computing Machinery. doi:10.1145/3465084.3467906.
  • [11] Soma Chaudhuri. More choices allow more faults: Set consensus problems in totally asynchronous systems. Inf. Comput., 105(1):132–158, July 1993. doi:10.1006/inco.1993.1043.
  • [12] Michael J. Fischer, Nancy A. Lynch, and Michael S. Paterson. Impossibility of distributed consensus with one faulty process. J. ACM, 32(2):374–382, April 1985. doi:10.1145/3149.214121.
  • [13] Eli Gafni, Sergio Rajsbaum, and Maurice Herlihy. Subconsensus tasks: Renaming is weaker than set agreement. In Proceedings of the 20th International Conference on Distributed Computing, DISC’06, pages 329–338, Berlin, Heidelberg, 2006. Springer-Verlag. doi:10.1007/11864219_23.
  • [14] Maurice Herlihy and Nir Shavit. The topological structure of asynchronous computability. J. ACM, 46(6):858–923, November 1999. doi:10.1145/331524.331529.
  • [15] Gunnar Hoest and Nir Shavit. Towards a topological characterization of asynchronous complexity. In Proceedings of the Sixteenth Annual ACM Symposium on Principles of Distributed Computing, PODC ’97, pages 199–208, New York, NY, USA, 1997. Association for Computing Machinery. doi:10.1145/259380.259440.
  • [16] Shihao Liu. The Impossibility of Approximate Agreement on a Larger Class of Graphs. In Eshcar Hillel, Roberto Palmieri, and Etienne Rivière, editors, 26th International Conference on Principles of Distributed Systems (OPODIS 2022), volume 253 of Leibniz International Proceedings in Informatics (LIPIcs), pages 22:1–22:20, Dagstuhl, Germany, 2023. Schloss Dagstuhl – Leibniz-Zentrum für Informatik. doi:10.4230/LIPIcs.OPODIS.2022.22.
  • [17] Michael Saks and Fotios Zaharoglou. Wait-free k-set agreement is impossible: The topology of public knowledge. SIAM J. Comput., 29(5):1449–1483, March 2000. doi:10.1137/S0097539796307698.
  • [18] Yusong Shi and Weidong Liu. Colorless tasks and extension-based proofs, 2024. arXiv:2303.14769.