Deciding Termination of Simple Randomized Loops
Abstract
We show that universal positive almost sure termination (UPAST) is decidable for a class of simple randomized programs, i.e., it is decidable whether the expected runtime of such a program is finite for all inputs. Our class contains all programs that consist of a single loop, with a linear loop guard and a loop body composed of two linear commuting and diagonalizable updates. In each iteration of the loop, the update to be carried out is picked at random, according to a fixed probability. We show the decidability of UPAST for this class of programs, where the program’s variables and inputs may range over various sub-semirings of the real numbers. In this way, we extend a line of research initiated by Tiwari in 2004 into the realm of randomized programs.
Keywords and phrases:
decision procedures, randomized programs, linear loops, positive almost sure terminationCopyright and License:
2012 ACM Subject Classification:
Theory of computation Probabilistic computation ; Theory of computation Program analysisSupplementary Material:
Software (Source Code): https://github.com/aprove-developers/SiRop [24]
archived at
swh:1:dir:3ca664cebef79bfeb95ec944ddc8441d3b528bf6
Funding:
DFG Research Training Group 2236 UnRAVeL.Acknowledgements:
We thank Sophia Greiwe for her help with the implementation of our decision procedure in SiRop.Editors:
Paweł Gawrychowski, Filip Mazowiecki, and Michał SkrzypczakSeries and Publisher:
Leibniz International Proceedings in Informatics, Schloss Dagstuhl – Leibniz-Zentrum für Informatik
1 Introduction
We consider the problem of universal positive almost sure termination (UPAST), i.e., deciding whether a given randomized program has finite expected runtime on all inputs [6, 34]. This is a stronger property than universal almost sure termination (UAST) which requires that the probability of termination is . Our programs are simple randomized loops of the form
| (1) |
Here, denotes the vector of program variables that range over a semiring , and is a matrix representing the loop guard with linear constraints over the program variables. In each execution of the loop body, a matrix is chosen among according to the probability and the value is updated accordingly.
Our Contribution.
We show that UPAST is decidable for all when limited to loops with diagonalizable commuting matrices and , where is the set of algebraic real numbers.111Our approach only considers algebraic real , , , and , as it is not possible to represent arbitrary real numbers on computers. However, in Sect. 6 we will see that such a loop terminates for all algebraic real inputs iff it terminates for all real inputs. Thus, we extend previous results on the termination of linear and affine222In an affine (or non-homogeneous) loop, the guard may have the form and the update may have the form for arbitrary vectors and . non-randomized loops to the randomized setting. In addition to deciding universal termination, our approach can compute a non-termination witness , i.e., if the loop is non-terminating, then is an input leading to an infinite expected runtime.
Our programs go beyond single path loops as we might have . Thus, for every ,there is not just a single execution of length but one has a “range” of possible executions of length where each execution occurs according to a known probability. To ensure tractability of the resulting problem we require commutativity of both updates, so that we can focus on how often each update has been selected in an execution, but we do not have to take the different orders into account in which the two updates might have been chosen. Moreover, we require diagonalizability to obtain closed forms of a certain shape, which allows us to analyze the behavior of a “range” of different executions at once. To demonstrate the practical applicability of our decision procedure and the computation of non-termination witnesses, we provide a prototype implementation for the case with our tool SiRop.
Related Work.
We continue a line of research started in 2004 by Tiwari [35] who showed decidability of universal termination for loops with an affine guard and an affine update as its body, where the guard, updates, and inputs range over the real numbers. In his proof, Tiwari reduced the affine to the linear case. In 2006, Braverman [7] proved that the problem remains decidable for loops and inputs ranging over the rational numbers , and if the guard and update are linear, then he also showed decidability over the integers . Similar to Tiwari, Braverman also reduced the affine case for to the linear case. In 2015, Ouaknine et al. proved [31] that the affine case is decidable over the integers whenever the update is of the form , provided that is diagonalizable. This restriction was removed by Hosseini et al. in 2019 [18]. In a related line of work, we proved decidability of universal termination over the integers for triangular affine loops, i.e., where the matrix is triangular [12]. Later, we extended these results to triangular weakly non-linear loops which extend triangular loops by allowing certain non-linear updates [15, 16].
The only decidability results for termination of randomized programs that we are aware of consider probabilistic vector addition systems [8] or constant probability programs [13], i.e., loops whose guards consist of only one affine inequation and whose bodies consist of several probabilistic branches (with fixed probabilities) that may increase or decrease the program variables by fixed constants. The programs in [8, 13] are orthogonal to the ones considered in our approach as they only allow to modify the program variables by adding constants, but do not allow for multiplication. Another related area of research [4, 21, 28] deals with prob-solvable loops and moment invariants. Given such a loop, these techniques can compute closed forms for all moments of program variables for a given iteration of the loopand, by taking a limit, also upon the loop’s termination. Thus, if restricted to almost surely terminating programs, they can decide UPAST. However, in contrast to our method, these techniques require that all variables in the loop guard may only take finitely many values. Moreover, there are many automated approaches for tackling UPAST using so-called ranking supermartingales (RSM), e.g., [1, 3, 5, 9, 10, 25, 27, 30, 36]. To generate a suitable RSM, one often uses techniques based on affine or polynomial templates, which renders the approach incomplete. In [19], the authors showed that deciding UPAST is harder than deciding universal termination for non-randomized programs in terms of the arithmetic hierarchy.
Outline of our Approach.
We assume familiarity with basics from probability theory and linear algebra (we recapitulate some main concepts in [26]). Sect. 2 formally introduces simple randomized loops, gives their semantics in terms of a probability space, and presents most of the definitions used throughout the paper. For a loop as in 1, we consider (finite) executions corresponding to words over the alphabet , where the -th symbol in indicates which update matrix was used in the -th application of the assignment . For such executions , and denote the number of - and -symbols in , respectively. Moreover, we introduce the function that maps finite executions to the values of the constraints in the loop guard after executing on a concrete input , i.e., , since and commute. Our decision procedure does not search for a non-terminating input directly, but for an eventually non-terminating input . An input is eventually non-terminating if by repeated execution of the loop body on (while ignoring the guard), a non-terminating input can eventually be reached. In Lemma 12, we show that a loop has an eventually non-terminating input iff it also has a non-terminating input. Later (in Sect. 6) we will show how to lift such an eventually non-terminating input to an actual non-terminating input.
In Sect. 3, we introduce a mapping that maps executions to the difference between the relative number of times that the update matrix has been chosen in and the probability of choosing , i.e., . Moreover, we essentially partition the set of indices of all program variables into suitable sets with for some finite set . We will then show that for all , and all executions with , the value of the -th constraint after executing on is
| (2) |
Here, all are complex numbers of modulus , i.e., for all , and the functions are linear maps . The maps and the values only depend on the matrices , , and , but not on the specific input . While weaker requirements would suffice to ensure that has some closed form, diagonalizability of and guarantees that it has the form (2), which is crucial for our procedure. Lemma 21 shows that by a lexicographic comparison of those for which the inner sum in 2 is not for all executions , one can compute which of the pairs is the “dominant” one. Here, a pair is considered dominant whenever the value of the first factor of 2 grows the fastest if the execution of is continued (i.e., if ) and the corresponding inner sum is not for all executions . The dominant pair depends on the specific input and on whether is ositive or egative, and correspondingly, one has to use different lexicographic comparisons to determine the dominant pair. For , let denote the set where the pair is dominant for input and (and positive if or negative for ). Then, the sign of the “coefficient” of the dominant pair eventually determines the sign of , provided that is large enough (Lemma 23).
In Sect. 4, we consider the rearrangement
| (3) |
where and .
Lemma 28 then gives a necessary condition for non-termination (and hence a sufficient condition for universal termination): If is an eventually non-terminating input, then there must be a such that we have for all constraints , with as in 3.
Sect. 5 turns this necessary condition for non-termination into a sufficient condition. To that end, we define the set of witnesses for eventual non-termination containing all inputs for which there is some such that holds for all constraints . Lemma 31 shows that all are eventually non-terminating. While this condition is only sufficient for non-termination, we show in Lemma 32 that if the program is non-terminating, then there is also an input in . So the considered program is non-terminating iff , i.e., the program is universally terminating iff (Cor. 33).
Finally, our decision procedure for UPAST is presented in Sect. 6. Lemma 34 shows that is semialgebraic and thus the emptiness problem is decidable over the real algebraic numbers, i.e., if . Moreover, Lemma 36 shows that can be represented as a finite union of convex semialgebraic sets. Hence, emptiness of can also be decided over the rationals and integers [20]. Cor. 39 shows how witnesses for non-termination can be obtained from eventually non-terminating inputs . We discuss our implementation in the tool SiRop in Sect. 7. For all proofs, we refer to [26]. Moreover, [26] also contains proof sketches for our main results to help understanding the essential proof ideas.
2 Programs & Termination
As usual, let denote the set of algebraic numbers, i.e., the set of all roots of (univariate) polynomials from . As mentioned, denotes the set of algebraic reals, and denotes the set of positive natural numbers below and including for every with , i.e., for and .
We now define our class of simple randomized loops. The program variables range over a semiring with a guard consisting of a conjunction of strict linear inequations over the program variables, represented by a matrix , and two commuting linear updates that are diagonalizable (over ). In each loop iteration, the applied update is chosen among according to the outcome of a (possibly biased) coin toss.
It is well known (e.g., [17, Thm. 1.3.12]) that two matrices for are commuting and diagonalizable iff they are simultaneously diagonalizable, i.e., there is a regular matrix such that and , where and are complex diagonal matrices, and and are the eigenvalues of and , respectively. Moreover, if are algebraic, then can also be chosen to be algebraic.
Definition 1 (Simple Randomized Loops).
Let , , , and such that and are simultaneously diagonalizable.333In the following, we assume and exclude the trivial semiring as every simple randomized loop terminates for the input . Then,
is called a simple randomized loop (over ) of dimension with constraints. In the remainder, we will omit “simple randomized” and just refer to these programs as “loops”.
The meaning of “” is that is updated to with probability and to with probability . The comparison is understood componentwise, i.e., iff for all , where is the -th entry of the vector . To simplify the notation, from now on we will consider a fixed loop of dimension with constraints. Moreover, w.l.o.g. we assume , i.e., , as otherwise one can set to if and to if , and then replace by an arbitrary number from .
A run of the loop is an infinite word , where Runs denotes the set of all runs. For a run , the value indicates which of the updates or was used in the -th iteration of the loop. Here, and are distinct markings even when . To simplify the notation, we introduce random variables for that map a run to its -th element. Note that all suchrandom variables are independent and identically distributed. A (finite) execution is a (possibly empty) prefix of a run. Let Path denote the (countable) set of all such finite executions. Given a finite execution with , let denote its length. Furthermore, for , denotes the number of performed updates with update matrix or , respectively, during the execution of .
Since the definition of runs is independent from the specific input of the loop, the semantics of the loop depend only on the value of . To obtain a probability measure for , one first considers cylinder sets for all , i.e., contains all runs with prefix . By requiring for all , one obtains a (unique) probability measure on the -field generated by all cylinder sets , see, e.g., [2, Thm. 2.7.2].
Definition 2 (Semantics of Loops).
The semantics of is given as a probability space where and .
To capture the behavior of on some specific input , we introduce a function that associates finite executions with the values of ’s guard after the execution of . Recall that and commute. Hence, we define
where for every matrix , is the -dimensional identity matrix.
Lemma 3 (Values of Constraints).
For any and we have
Example 4.
Consider the loop “” with
Then, for and all , equals
The value of the -th constraint after the execution of on the initial value is given by the expression
| (Lemma 3) | ||||
| (4) |
for linear maps with
| (5) |
Here, denotes the entry of at row and column . Moreover, since , , and , we have for all .444This observation will be needed in the final SMT encoding for our decision procedure (see Lemma 34), as we have to encode the coefficients for a given . In the following, we refer to the addends of the sum in 4 as constraint terms.
Example 5.
Reconsider Ex. 4. Then, with eigenvalues , and
Corollary 6.
For all and all , due to the definition of we have and .
The following lemma shows that if one has two pairs of eigenvalues and where and are the complex conjugates of and , then the sum of all linear maps where is the complex conjugate of the sum of all linear maps where . For instance in Ex. 5, are the complex conjugates of and indeed, we have . This lemma will later be needed to show that when representing and summing up the coefficients of its addends in a suitable way, all resulting coefficients are real numbers (see Remark 18).
Lemma 7 (Sums of Conjugated Constraint Terms are Real-Valued).
Let , let , and let be the linear map from 5. Then, for all inputs we have where
In order to define the notion of termination for , we first introduce the concept of a run’s length by counting the number of iterations until the guard is violated for the first time. Throughout the paper, we use the convention .
Definition 8 (Length Of Runs).
For any , we define the random variable as .
We now define the expected runtime of for the input as the expectation . If , we call the corresponding input non-terminating. So we consider positive almost sure termination [6, 34], where termination corresponds to a finite expected runtime.
Definition 9 (Non-Terminating Inputs).
The set of non-terminating inputs is .
Consequently, we call terminating whenever and non-terminating otherwise.
As in [7, 15, 16, 18, 31, 35], we focus on eventual non-termination instead of actual non-termination as this allows us to ignore a finite number of initial updates of the loop. In our setting, an input is eventually non-terminating if a non-terminating input can be reached by repeated application of the updates in the loop body to .
Definition 10 (Eventual Non-Termination).
We define the set of eventually non-terminating inputs as .
The motivation behind considering instead of is that it allows us to “jump” over the first iterations of the loop (where the loop guard might be violated). In this way, we can focus only on the longterm behavior of the loop on a given input.
Example 11 (Difference Between & ).
Consider the loop “” with and . Then, as , i.e., violates the loop guard. While also violates the loop guard (since ), we have for all . Thus, and therefore, .
Considering instead of is justified by the fact that iff , as shown by Lemma 12.
Lemma 12 (Correspondence of & ).
For any semiring , we have iff .
3 On Constraint Terms
In this section we consider the value for , motivated by our interest in eventual non-termination. In the first part of this section, we represent , for , as a sum over so-called “constraint term groups”, expressed using a quantity corresponding to “how much has deviated from the expected execution”. The section’s second part then shows that for specific it suffices to only consider certain addends of this sum in order to decide whether as . This observation will lead to a necessary condition for eventual non-termination in Sect. 4, which will subsequently be turned into a sufficient criterion in Sect. 5.
When executing the loop , the relative number of times that update is selected over will intuitively tend towards with increasing number of iterations. We now consider this relative quantity and additionally subtract to center its distribution around .
Definition 13 (Deviation From Equilibrium).
The mapping is defined as for every non-empty and otherwise.
We will investigate which addends determine the sign of for . To this end, we want to express the value of the constraint terms after some finite execution in terms of and . The advantage is that for sufficiently long paths , we know how and “behave” (i.e., “tends towards” and is “expected to tend towards” 0).
Lemma 14 (Normal Form of Constraint Terms).
Let be a constraint term with . We write in polar form as and , respectively, where are complex units, i.e., . Then, for any (non-empty) finite execution with we have
Note that in Lemma 14 we excluded all constraint terms with or in order to avoid a division by zero. However, we additionally required555The set of runs where for every prefix of has probability 0, as means that only or has been selected in . However, we had required . implying and , since . Hence, for all constraint terms with or and all considered , we have and thus the value of such constraint terms can safely be ignored when computing . This leads to the equation
| (6) |
where and .
By inspecting the right-hand side of 6 it becomes clear that the subexpressions and govern the overall asymptotic growth of as increases. In the following, we group all constraint terms into so-called constraint term groups which are sets of indices corresponding to constraint terms where and have common values and . Here, is the expression that is important in 6 if is in a region close to 0.666Note that is the -th power of the weighted geometric mean whenever . If one is outside such a region, then is important as well.
Definition 16 (Constraint Term Groups).
For any , let
Moreover, we define the finite set of all pairs with . For and , let whenever holds for all .777We will show how to check this in Lemma 24. Note that this is not implied by . As a counterexample, consider , (and thus, ), and , (and hence, ). Otherwise let . We refer to the sets as constraint term groups.
Lemma 17 shows that for all we have and iff and . Thus, if , then and . We will first return to this result in Sect. 4, where we will use that for all with we have . Later on, we will revisit it in Sect. 5.
Lemma 17 (Equality of Eigenvalues).
Let be positive reals such that and . Then we have .
Due to 7, we have to consider sums for and . While , , and are complex numbers in general, such sums are always real-valued. The corresponding Remark 18 is an immediate consequence of Lemma 7. Later, this remark will allow us to make statements about the signs of such sums, see Lemma 23.
Remark 18 (Coefficients of Constraint Term Groups are Real-Valued).
Let and . Then, for all we have .
Example 19.
We continue Ex. 15. There are two different non-empty sets , i.e., and . As indicated by Lemma 17, this implies and . Hence, and similarly, . Moreover, recall that . Therefore, we have
For the last step, the direction “” is clear, since implies for all . The direction “” is also clear by choosing to be the empty path.
Hence, . Similarly, for .
By inspecting 7 again, one observes that if is sufficiently close to , then the terms belonging to the non-empty constraint term group with maximal in the lexicographic ordering will at some point (when ) outgrow all other terms whenever is positive and sufficiently large. If is negative and sufficiently small, then, however, we have to focus our attention on the non-empty group for which is maximal in the lexicographic ordering.888Since and are always positive reals, the expressions determine whether is positive or negative. However, for , this expression (and also ) could alternate between being positive, negative, or even 0. This will be regarded in Sect. 4 and 5.
Definition 20 (Eventually Dominating Constraint Term Group).
Let or both denote the usual lexicographic ordering on and let be the lexicographic ordering where the comparison on the second component is flipped.
For all , , and we define if for all , and for otherwise, where denotes the maximum w.r.t. the ordering .
It is not a priori clear, how, for a constraint index and a given input , the sets can be computed automatically since one has to decide whether where are assumed to be positive algebraic reals, but and are in general non-algebraic reals. This is due to the well-known Gelfond-Schneider theorem (see, e.g., [29, Thm. 3.0.1]), which states that whenever and is irrational. However, Lemma 21 ensures the decidability of such comparisons.
Lemma 21 (Comparing Constraint Term Groups).
Let and for positive algebraic reals and . Then the statement is decidable.
Example 22.
Recall the two non-empty constraint term groups and from Ex. 19. Then, for , we have
Lemma 23 states the main property of the eventually dominating constraint term groups and . Fig. 1 depicts this lemma (and also the following lemmas) graphically. Here, the horizontal axis represents the length of the path and the vertical axis represents the value of the function , which expresses the deviation of the number of -symbols in the execution from the expected number of -symbols. We depicted by a gray line. The lemma essentially states that whenever reaches one of the two “safe” regions marked in green, then the coefficient of the dominant addend determines the sign of , provided that its absolute value is large enough. The upper safe region is the one for , i.e., here the path is long enough (i.e., ), (i.e., ), and . Similarly, the lower safe region corresponds to the case . This lemma also indicates why an extension of our approach to programs with three instead of two update matrices would be problematic. Then instead of we would need a vector to express how much an execution deviates from the probabilities in the program. This would break our concepts of eventually dominating constraint term groups and safe regions, since instead of , we would have to consider the “direction” of this deviation.
Lemma 23 (Domination of Eventually Dominating Constraint Term Groups).
Let , , and . We define (see Remark 18) as
Then for every , there exist constants , , and (for a bound on the length of the path), such that for all with , , , if , and if , we have .
4 Positive Eigenvalues
Recall that we are interested in as the execution progresses, i.e., for . By Lemma 23, to this end we have to consider the sign of , where and . If both and are positive reals, then for , the sign of does not change. Thus, we now investigate restricted to all where or is not from . In Lemma 24, we will show that this sum is either always 0 (for all paths ) or it becomes negative for large enough .
Assume that for some constraint , , and input , some eigenvalue of each constraint term in the eventually dominating constraint term group is not positive real, i.e., for all one has or . Then, the sign of the real part of this constraint term will change throughout the program’s execution (i.e., for ). Lemma 24 shows that if the sum of these constraint terms is not always 0, then irrespective of the updates that were already performed in previous iterations, this sum becomes smaller than some negative constant after a number of further iterations. This is expressed in Lemma 24(b), where we have already performed updates with the matrix and updates with the matrix . Then by extending the run long enough, the real part of the sum becomes smaller than a constant that does not depend on and . Our Lemma 24 is a generalization of a similar result by Braverman [7, Lemma 4] to products of orbits of complex units, i.e., to products of for .999Note that Lemma 24 allows us to check whether holds for all , see Footnote 7. By Remark 18, the sum is a real number and thus, in the case of (a), holds for all . So given an actual input , one just has to check the condition of Lemma 24(a). If that condition does not hold, then by Lemma 24(b), does not hold for every .
Lemma 24 (Coefficients of Complex Eigenvalues Become Negative).
Let be complex numbers and let be complex units such that or for all . For all , let . If all tuples for are pairwise different, then there exist constants and such that we either have (a) or (b):
-
(a)
For all with there is some with , , and , which implies for all .
-
(b)
For all there exist such that and there are such that .
Example 25.
Let be the set of indices such that both eigenvalues and are positive reals and let be the set where at least one of the eigenvalues is not a positive real. To simplify the notation we also denote and by and , respectively, for . So for all , we have , i.e., the sign of the corresponding addend does not change for . For the other eigenvalues, by Lemma 24, is either always 0 (for all paths ) or it becomes negative for suitable and .
When executing the loop on input , one expects that eventually (for ) the constraint term group for either or dominates the sign of constraint (Lemma 23). Whenever and has reached one of the two “safe” regions marked in green in Fig. 1, by Lemma 24 one can extend the current path by a path such that the coefficient of the dominating constraint term group is negative. Thus, the execution can be extended by a path such that it leads to termination. This observation is captured in Lemma 26.
Lemma 26 (Finite Execution leading to Termination).
Let , , and , such that . Then there are constants , , and , such that for all with , , if , and if , there is a finite execution of length with .
Finally, Lemma 28 builds upon Lemma 26 and gives a sufficient criterion for termination of an input . The negation of this criterion is a necessary criterion for every input that is eventually non-terminating. This necessary criterion states that if is eventuallynon-terminating, then for all constraints , the sum of the addends for the “dual positive eigenvalues” (where both and are positive reals) must be positive.
So whenever , Lemma 28 states that the expected number of steps until reaches a “safe” (green) area in Fig. 1 and executes afterwards is finite. In other words, the expected number of steps until termination is finite.
Example 27.
To motivate Lemma 28 further, we continue Ex. 22. Let . We have , as only and are positive real eigenvalues.
First, suppose and . Then, for all we have and hence . Thus, by Lemma 28.
Lemma 28 (Dual Positive Eigenvalues for Eventually Dominating Constraints).
Let . If for every there is a with , then . Thus, if , then there is some such that for all we have .
5 Towards Non-Termination Witnesses
Lemma 28 provides a necessary condition that must hold for all . It requires that thesum of the addends for all positive real eigenvalues must be . This condition is however not sufficient for . To turn this into a sufficient criterion, we now increase the lower bound . More precisely, we replace by the sum of the addends for all those eigenvalues where or are not a positive real number. In this way, we obtain a sufficient (but no longer necessary) criterion for . To turn this into a sufficient and necessarycriterion, we then introduce a “boosting lemma” (Lemma 32), which states that if there is aninput in , then there is also a (possibly different) input in that satisfies our sufficient criterion. To prove this boosting lemma, we need the necessary condition of Lemma 28.
For our sufficient (but not necessary) condition for , we define the set of witnesses for eventual non-termination as those inputs meeting this criterion.
Definition 29 (Witnesses for Eventual Non-Termination).
We define the set of witnesses for eventual non-termination, where for , we have
Note that the sum on the left-hand side in the definition of is real-valued due to Lemma 7.
So are all inputs where for all constraints , the sum of the dominating addends for positive real eigenvalues is greater than the sum of the for the other eigenvalues . Lemma 31 shows that the witness condition of Def. 29 is indeed a sufficient criterion for . Before presenting this lemma, we will apply it to our running example.
Example 30.
Lemma 31 (Witness Criterion is Sufficient for Eventual Non-Termination).
Let be a witness for eventual non-termination, i.e., . Then we have , i.e., is indeed an eventually non-terminating input.
Def. 29 introduced a set that, as shown by Lemma 31, under-approximates the set of eventually non-terminating inputs . While in general we may have , for the program from Ex. 30 we have as for every one either has or . So here, implies for some by Lemma 28 and hence , i.e., .
As the set is rather simple to characterize in contrast to , our goal is to only check for the existence of some . This input then witnesses the eventual non-termination of the loop . The following Lemma 32 establishes that whenever is eventually non-terminating, then such a witness does indeed exist. This then leads to our overall decision procedure, because we have that is non-terminating is eventually non-terminating , see Cor. 33. In Sect. 6, we will show that emptiness of is decidable (not only over the algebraic reals, but also over different sub-semirings of such as the naturals, integers, or rationals) and if , then an element of is computable.
The intuition behind Lemma 32 is as follows: Given an input , we want to construct a non-terminating input in . Recall that for any constraint and , the set contains those indices from the dominant constraint term group where the corresponding eigenvalues and of both update matrices and are positive reals. On the other hand, contains the remaining indices from the dominant constraint term group. Moreover, the help to determine the sign of the corresponding dominant pair’s coefficient. If , then
| (8) |
for some . One can now modify to make 8’s left-hand side smaller. For every , if is not positive real, then we multiply by , and otherwise by . Since we have by Cor. 6, this “shifts” the phase of at least on the left-hand side, but not for the addends on the right-hand side of 8. By performing such multiplications repeatedly and taking a linear combination of the obtained inputs, 8’s left-hand side becomes arbitrarily small since addends “cancel out”, whereas this is not the case for the right-hand side. So one obtains a non-terminating input where 8 does not hold for any . Thus, .
Lemma 32 (Boosting).
Let be a non-terminating loop over , i.e., and . Then there is a corresponding witness in .
The following corollary summarizes our results so far, i.e., it shows that non-termination is equivalent to the existence of an element in .
Corollary 33 (Characterizing Termination).
A loop is terminating over a semiring iff .
6 Deciding PAST
Finally, we present our novel technique for deciding whether a loop is (positively almost surely) terminating, i.e., whether its expected runtime is finite for every input. As discussed in Sect. 5, to this end we only have to show decidability of for the set of witnesses for eventual non-termination from Def. 29. We now explain how to translate this emptiness problem into an SMT problem. More precisely, we show that the witness set is semialgebraic, i.e., it corresponds to a formula over polynomial arithmetic (which is linear in the variables ). For this we have to take into account that for different values of , different addends may be eventually dominating. Then, decidability over the algebraic reals is clear.
As before, are those indices from where both eigenvalues and are positive reals, and are the remaining indices.
Lemma 34 (Semialgebraic Sets of Witnesses for Algebraic Loops).
Let , . We define the sets as
for all . Then, for all and we define as
Then
| (9) |
Furthermore, we have . The sets and the set are moreover semialgebraic.
Example 35.
We continue Ex. 30 and consider . By Ex. 30 we have . Moreover, there is no with . For this implies .
Thus, the loop initially introduced in Ex. 4 is non-terminating for all .111111We did not consider as for such a choice of we do not have .
Note that while for the number is not necessarily algebraic, the representation of as a finite union/intersection of the semialgebraic sets is still computable by Lemma 21 as one simply has to determine the corresponding ordering on . This is the reason why we restricted to the set of algebraic reals.
To show that emptiness of is also decidable over various sub-semirings of the algebraic reals, we prove the convexity of the sets . Note that the set as well as the sets themselves are in general not convex.
Lemma 36 ( as Finite Union of Convex Sets).
For and , the set is convex, i.e., for all and .
Note that Lemmas 34 and 36 imply that for the set is semialgebraic and a finite union of convex sets.
Theorem 37 (Deciding PAST).
Let . Then, the question whether a loop is terminating on is decidable, and if the loop is non-terminating, then a witness for eventual non-termination can be computed.
Remark 38.
The theory of the reals and the algebraic reals are elementary equivalent as both are real closed fields. Thus, Thm. 37 directly entails that the question whether there exists a non-terminating non-negative real input or real input for an algebraic loop is decidable as well, if one extends the set and the corresponding definitions to real inputs . Note that in this case iff .
While the procedure outlined in the proof of Thm. 37 only allows for the computation of a witness for eventual non-termination, one can lift this to the computation of a witness according to the constructive proofs of Lemmas 23 and 31.
Corollary 39 (Computing Witnesses for Non-Termination).
Let . If a loop is non-terminating, then a witness for non-termination from can be computed.
7 Implementation and Conclusion
Prototype Implementation.
To demonstrate the practical applicability of our decision procedure, we implemented it in our prototype tool SiRop (for “Simple Randomized Loops”). The tool and a corresponding collection of exemplary programs can be obtained from
The tool is implemented in Python and uses the SageMath open-source mathematics software system [33] in order to perform necessary computations such as simultaneous diagonalization and determining the mappings . SiRop tries to compute a witness for eventual non-termination by creating a corresponding SMT problem which is then solved using the SMT-RAT [11] solver. If the SMT problem is unsatisfiable, then the program is terminating. In contrast, if such a witness is found, then the program is non-terminating and the tool computes a non-terminating input from . Currently, SiRop handles loops over the algebraic reals only, i.e., , as for all other considered sub-semirings of , the decision procedure relies on the technique presented in [20] which (to the best of our knowledge) has not yet been implemented.
Conclusion.
We have shown the decidability of universal positive almost sure termination (UPAST) for the class of simple randomized loop ranging over numerous semirings , thereby transferring a line of research started in 2004 by Tiwari [35] on universal termination of linear loops to the realm of randomized programs. To that end, we devised a corresponding decision procedure and presented a prototype implementation for the case , showing the practical applicability of the presented approach. In particular, our tool managed to find a non-terminating algebraic input for one121212https://github.com/TermCOMP/TPDB/blob/11.3/C_Integer/Stroeder_15/ChenFlurMukhopadhyay-SAS2012-Ex2.06_false-termination.c of the only two problems from the category C Integer which were not solved by any tool at the 2023 Termination Competition [14],131313This category was not part of the 2024 competition. the other one being the Collatz problem. While our tool only considers (whereas the problem is formulated over the integers), the constraints generated by SiRop are unsatisfiable over , which implies universal termination of the program over the integers.
Future Work.
While our procedure can decide positive almost sure termination for all inputs, in the future we want to improve it such that it can also compute bounds on expected runtimes. Moreover, decision procedures for termination or complexity of subclasses of non-randomized programs (e.g., [12, 15, 16, 32]) have been integrated in (incomplete) tools that analyze general programs [22, 23], and we would like to investigate such an integration for randomized programs as well. Finally, we plan to adapt our approach to a decision procedure for universal almost sure termination (UAST), i.e., whether a program terminates with probability on all inputs. Clearly, UPAST implies UAST but the converse does not hold in general.
References
- [1] Sheshansh Agrawal, Krishnendu Chatterjee, and Petr Novotný. Lexicographic ranking supermartingales: An efficient approach to termination of probabilistic programs. Proc. ACM Program. Lang., 2(POPL):34:1–34:32, 2018. doi:10.1145/3158122.
- [2] Robert B. Ash and Catherine A. Doléans-Dade. Probability and Measure Theory. Academic Press, 2nd ed. edition, 2000.
- [3] Martin Avanzini, Georg Moser, and Michael Schaper. A modular cost analysis for probabilistic programs. Proc. ACM Program. Lang., 4(OOPSLA), 2020. doi:10.1145/3428240.
- [4] Ezio Bartocci, Laura Kovács, and Miroslav Stankovic. Automatic generation of moment-based invariants for prob-solvable loops. In Proc. ATVA ’19, LNCS 11781, pages 255–276, 2019. doi:10.1007/978-3-030-31784-3_15.
- [5] Kevin Batz, Mingshuai Chen, Sebastian Junges, Benjamin Lucien Kaminski, Joost-Pieter Katoen, and Christoph Matheja. Probabilistic program verification via inductive synthesis of inductive invariants. In Proc. TACAS ’23, LNCS 13994, pages 410–429, 2023. doi:10.1007/978-3-031-30820-8_25.
- [6] Olivier Bournez and Florent Garnier. Proving positive almost-sure termination. In Proc. RTA ’05, LNCS 3467, pages 323–337, 2005. doi:10.1007/978-3-540-32033-3_24.
- [7] Mark Braverman. Termination of integer linear programs. In Proc. CAV ’06, LNCS 4144, pages 372–385, 2006. doi:10.1007/11817963_34.
- [8] Tomás Brázdil, Krishnendu Chatterjee, Antonín Kucera, Petr Novotný, and Dominik Velan. Deciding fast termination for probabilistic VASS with nondeterminism. In Proc. ATVA ’19, LNCS 11781, pages 462–478, 2019. doi:10.1007/978-3-030-31784-3_27.
- [9] Aleksandar Chakarov and Sriram Sankaranarayanan. Probabilistic program analysis with martingales. In Proc. CAV ’13, LNCS 8044, pages 511–526, 2013. doi:10.1007/978-3-642-39799-8_34.
- [10] Krishnendu Chatterjee, Hongfei Fu, and Amir Kafshdar Goharshady. Termination analysis of probabilistic programs through positivstellensatz’s. In Proc. CAV ’16, LNCS 9779, pages 3–22, 2016. doi:10.1007/978-3-319-41528-4_1.
- [11] Florian Corzilius, Gereon Kremer, Sebastian Junges, Stefan Schupp, and Erika Ábrahám. SMT-RAT: an open source C++ toolbox for strategic and parallel SMT solving. In Proc. SAT ’15, LNCS 9340, pages 360–368, 2015. doi:10.1007/978-3-319-24318-4_26.
- [12] Florian Frohn and Jürgen Giesl. Termination of triangular integer loops is decidable. In Proc. CAV ’19, LNCS 11562, pages 426–444, 2019. doi:10.1007/978-3-030-25543-5_24.
- [13] Jürgen Giesl, Peter Giesl, and Marcel Hark. Computing expected runtimes for constant probability programs. In Proc. CADE ’19, LNCS 11716, pages 269–286, 2019. doi:10.1007/978-3-030-29436-6_16.
- [14] Jürgen Giesl, Albert Rubio, Christian Sternagel, Johannes Waldmann, and Akihisa Yamada. The termination and complexity competition. In Proc. TACAS ’19, LNCS 11429, pages 156–166, 2019. Website of the Annual Termination Competition: https://termination-portal.org/wiki/Termination_Competition. doi:10.1007/978-3-030-17502-3_10.
- [15] Marcel Hark, Florian Frohn, and Jürgen Giesl. Polynomial loops: Beyond termination. In Proc. LPAR ’20, EPiC 73, pages 279–297, 2020. doi:10.29007/nxv1.
- [16] Marcel Hark, Florian Frohn, and Jürgen Giesl. Termination of triangular polynomial loops. Formal Methods in Syst. Des., 65(1), 2025. doi:10.1007/s10703-023-00440-z.
- [17] Roger A. Horn and Charles R. Johnson. Matrix Analysis. Cambridge University Press, 2 edition, 2012. doi:10.1017/CBO9781139020411.
- [18] Mehran Hosseini, Joël Ouaknine, and James Worrell. Termination of linear loops over the integers. In Proc. ICALP ’19, LIPIcs 132, 2019. doi:10.4230/LIPIcs.ICALP.2019.118.
- [19] Benjamin Lucien Kaminski, Joost-Pieter Katoen, and Christoph Matheja. On the hardness of analyzing probabilistic programs. Acta Informatica, 56(3):255–285, 2019. doi:10.1007/S00236-018-0321-1.
- [20] Leonid Khachiyan and Lorant Porkolab. Computing integral points in convex semi-algebraic sets. In Proc. FOCS ’97, pages 162–171, 1997. doi:10.1109/SFCS.1997.646105.
- [21] Andrey Kofnov, Marcel Moosbrugger, Miroslav Stankovic, Ezio Bartocci, and Efstathia Bura. Exact and approximate moment derivation for probabilistic loops with non-polynomial assignments. ACM Trans. Model. Comput. Simul., 34(3):18:1–18:25, 2024. doi:10.1145/3641545.
- [22] Nils Lommen and Jürgen Giesl. Targeting completeness: Using closed forms for size bounds of integer programs. In Proc. FroCoS ’23, LNCS 14279, pages 3–22, 2023. doi:10.1007/978-3-031-43369-6_1.
- [23] Nils Lommen, Fabian Meyer, and Jürgen Giesl. Automatic complexity analysis of integer programs via triangular weakly non-linear loops. In Proc. IJCAR ’22, LNCS 13385, pages 734–754, 2022. doi:10.1007/978-3-031-10769-6_43.
- [24] Éléanore Meyer, Jürgen Giesl, and Sophia Greiwe. SiRop. Software, DFG-Research Training Group 2236 UnRAVeL, swhId: swh:1:dir:3ca664cebef79bfeb95ec944ddc8441d3b528bf6 (visited on 2025-08-05). URL: https://github.com/aprove-developers/SiRop, doi:10.4230/artifacts.24333.
- [25] Fabian Meyer, Marcel Hark, and Jürgen Giesl. Inferring expected runtimes of probabilistic integer programs using expected sizes. In Proc. TACAS ’21, LNCS 12651, pages 250–269, 2021. doi:10.1007/978-3-030-72016-2_14.
- [26] Éléanore Meyer and Jürgen Giesl. Deciding termination of simple randomized loops, 2025. doi:10.48550/arXiv.2506.18541.
- [27] Marcel Moosbrugger, Ezio Bartocci, Joost-Pieter Katoen, and Laura Kovács. The probabilistic termination tool Amber. Formal Methods Syst. Des., 61(1):90–109, 2022. doi:10.1007/S10703-023-00424-Z.
- [28] Marcel Moosbrugger, Miroslav Stankovic, Ezio Bartocci, and Laura Kovács. This is the moment for probabilistic loops. Proc. ACM Program. Lang., 6(OOPSLA2):1497–1525, 2022. doi:10.1145/3563341.
- [29] Saradha Natarajan and Ravindranathan Thangadurai. Pillars of Transcendental Number Theory. Springer, 2020. doi:10.1007/978-981-15-4155-1.
- [30] Van Chan Ngo, Quentin Carbonneaux, and Jan Hoffmann. Bounded expectations: Resource analysis for probabilistic programs. In Proc. PLDI ’18, pages 496–512, 2018. doi:10.1145/3192366.3192394.
- [31] Joël Ouaknine, João Sousa Pinto, and James Worrell. On termination of integer linear loops. In Proc. SODA ’15, pages 957–969, 2015. doi:10.1137/1.9781611973730.65.
- [32] Enric Rodríguez-Carbonell and Deepak Kapur. Automatic generation of polynomial loop invariants: Algebraic foundations. In Proc. ISSAC ’04, pages 266–273, 2004. doi:10.1145/1005285.1005324.
- [33] SageMath, the Sage Mathematics Software System (Version 10.3), 2024. https://www.sagemath.org.
- [34] Nasser Saheb-Djahromi. Probabilistic LCF. In Proc. MFCS ’78, LNCS 64, pages 442–451, 1978. doi:10.1007/3-540-08921-7_92.
- [35] Ashish Tiwari. Termination of linear programs. In Proc. CAV ’04, LNCS 3114, pages 70–82, 2004. doi:10.1007/978-3-540-27813-9_6.
- [36] Di Wang, David M. Kahn, and Jan Hoffmann. Raising expectations: Automating expected cost analysis with types. Proc. ACM Program. Lang., 4(ICFP), 2020. doi:10.1145/3408992.
