Infinite-Duration Bidding Games

Two-player games on graphs are widely studied in formal methods as they model the interaction between a system and its environment. The game is played by moving a token throughout a graph to produce an infinite path. There are several common modes to determine how the players move the token through the graph; e.g., in turn-based games the players alternate turns in moving the token. We study the {\em bidding} mode of moving the token, which, to the best of our knowledge, has never been studied in infinite-duration games. Both players have separate {\em budgets}, which sum up to $1$. In each turn, a bidding takes place. Both players submit bids simultaneously, and a bid is legal if it does not exceed the available budget. The winner of the bidding pays his bid to the other player and moves the token. For reachability objectives, repeated bidding games have been studied and are called {\em Richman games} \cite{LLPU96,LLPSU99}. There, a central question is the existence and computation of {\em threshold} budgets; namely, a value $t \in [0,1]$ such that if \PO's budget exceeds $t$, he can win the game, and if \PT's budget exceeds $1-t$, he can win the game. We focus on parity games and mean-payoff games. We show the existence of threshold budgets in these games, and reduce the problem of finding them to Richman games. We also determine the strategy-complexity of an optimal strategy. Our most interesting result shows that memoryless strategies suffice for mean-payoff bidding games.


Introduction
Two-player infinite-duration games on graphs are an important class of games as they model the interaction of a system and its environment. Questions about automatic synthesis of a reactive system from its specification [41] are reduced to finding a winning strategy for the "system" player in a twoplayer game. The game is played by placing a token on a vertex in the graph and allowing the players to move it throughout the graph, thus producing an infinite trace. The winner or value of the game is determined according to the trace. There are several common modes to determine how the players move the token that are used to model different types of systems (c.f., [4]). The most wellstudied mode is turn-based, where the vertices are partitioned between the players and the player who controls the vertex on which the token is placed, moves it. Other modes include probabilistic and concurrent moves.
We study a different mode of moving, which we refer to as bidding, and to the best of our knowledge, has never been studied for infinite-duration games. Both players have budgets, where for convenience, we have B 1 + B 2 = 1. In each turn a bidding takes place for the right to move the token. The players submit bids simultaneously, where a bid is legal if it does not exceed the available budget. Thus, a bid is a real number in [0, B i ], for i ∈ {1, 2}. The player who bids higher least 2/3 + 1. 25 . In other words, we are back to v 0 with a higher budget. By continuing like this, Player 1's budget increases by at least 0.25 whenever the game returns to v 0 , thus eventually his budget exceeds 0.75 and he draws the game to t as in the naive solution above. A similar argument shows that Player 1 wins with a budget of 1/3 + in v 2 , and showing that Player 2 wins from v 1 with 1/3 + and from v 2 with 2/3 + , is dual.
We introduce and study infinite duration bidding games with richer qualitative objectives as well as quantitative objectives. Parity games are an important class of qualitative games as the problem of reactive synthesis from LTL specifications is reduced to a parity game. The vertices in a parity game are labeled by an index in {0, . . . , d}, for some d ∈ N, and an infinite trace is winning for Player 1 iff the parity of the maximal index that is visited infinitely often is odd. The quantitative games we focus on are mean-payoff games. An infinite outcome has a value, which can be thought of as the amount of money that Player 1 pays Player 2. Accordingly, we refer to the players in a mean-payoff game as Maximizer (Max, for short) and Minimizer (Min, for short). The vertices of a mean-payoff game are labeled by values in Z. Consider an infinite trace π. The energy of a prefix π n of length n of π, denoted E(π n ), is the sum of the values it traverses. The mean-payoff value of π is lim inf n→∞ E(π n )/n. We are interested in cases where Min can guarantee a non-positive mean-payoff value. It suffices to show that he can guarantee that an infinite outcome π either has infinitely many prefixes with E(π n ) = 0, or that the energy is bounded, thus there is N ∈ N such that for every n ∈ N, we have E(π n ) ≤ N . We stress the point that there are two "currencies" in the game: a "monopoly money" that is used to determine who moves the token and which the players do not care about once the game ends, and the values on the vertices, which is the value that Min and Max seek to minimize and maximize, respectively. We illustrate mean-payoff games with the following example.
v 0 v 2 t v 1 1 −1 Figure 1 On the left, a bidding reachability game. On the right, a bidding mean-payoff game where the weights are depicted on the edges. Example 2. Consider the mean-payoff bidding game that is depicted in Figure 1, where for convenience the values are placed on the edges and not on the vertices. We claim that Min has a strategy that guarantees a non-positive mean-payoff value. Without loss of generality, Max always chooses the 1-valued edge. Min's strategy is a tit-for-tat-like strategy, and he always takes the (−1)valued edge. The difficulty is in finidng the right bids. Initially, Min bids 0. Assume Max wins a bidding with b > 0. Min will try and match this win: he bids b until he wins with it. Let b 1 , . . . , b n be Max's winning bids before Min wins with b. We call these un-matched bids. The next bid Min attempts to match is b = min 1≤i≤n b i ; he bids b until he wins with it, and continues similarly until all bids are matched.
We claim that the tit-for-tat strategy guarantees a non-positive mean-payoff value. Observe first that if a prefix of the outcome has k unmatched bids, then the energy is k. In particular, if all bids are matched, the energy is 0. Suppose Min bids b. We claim that the number of un-matched bids is at most 1/b . Otherwise, since b is less than all other un-matched bids, Max would need to invest more than a budget of 1. It follows that an infinite outcome that never reaches energy level 0 has bounded energy, thus the mean-payoff value is non-positive.
We study the existence and computation of threshold budgets in parity and mean-payoff bidding games. Also, we determine the strategy complexity that is necessary for winning. Recall that a winning strategy in a game typically corresponds to an implementation of a system. A strategy that uses an unbounded memory, like the tit-for-tat strategy above, is not useful for implementing. Thus, our goal is to find strategies that use little or no memory, which are known as memoryless strategies.
We show that parity bidding games are linearly-reducible to Richman games allowing us to obtain all the positive results from these games; threshold budgets exist, are unique, and computing them is no harder than for Richman games, i.e., the problem is in NP and coNP. We find this result quite surprising since for most other modes of moving, parity games are considerably harder than reachability games. The crux of the proof considers bottom strongly-connected components (BSCCs, for short) in the arena, i.e., SCCs with no exiting edges. We show that in a strongly connected bidding parity game, exactly one of the players wins with every initial budget, thus the threshold budgets of the vertices of a BSCC are in {0, 1}. If the vertex with highest parity in a BSCC is odd, then Player 1 wins, i.e., the threshold budgets are all 0, and otherwise Player 2 wins, i.e., the threshold budgets are all 1. We can thus construct a Richman game by setting the target of Player 1 to the BSCCs that are winning for him and the target of Player 2 to the ones that are winning for him. Moreover, we show that memoryless strategies are sufficient for winning in these games. Finally, motivated by the need to find systems of high quality, we ask whether Player 1 can not only win, but win in a prompt manner [34]. In Büchi games the goal is to visit an accepting vertex infinitely often, prompt winning amounts to guaranteeing a visit to an accepting vertex every k steps, for some constant k ∈ N. We show a negative result; under mild assumptions, Player 2 can guarantee arbitrarily long periods with no visits to accepting vertices with any initial budget.
We proceed to study mean-payoff bidding games. We adapt the definition of threshold values; we say that t ∈ [0, 1] is a threshold value for Min if with a budget that exceeds t, Min can guarantee a non-positive mean-payoff value. On the other hand, if Max's budget exceeds 1 − t, he can guarantee a positive mean-payoff value. We show that threshold values exist and are unique in mean-payoff bidding games. The crux of the existence proof again considers the BSCCs of the game. We show that in a strongly-connected mean-payoff bidding game, the threshold budgets are in {0, 1}, thus again either Min "wins" or Max "wins" the game. Moreover, this classification can be determined in NP and coNP, thus the complexity of solving bidding mean-payoff games coincides with Richman games. Our results for strongly-connected games are obtained by developing the connection that was observed in [37,36] between the threshold budget and the reachability probability in a probabilistic model on the same structure as the game. We show a connection between bidding mean-payoff games and one-counter 2.5-player games [14,13] to prove the classification of BSCCs. In turn, these games are equivalent to discrete quasi-birth-death processes [24] and generalize solvency games [11], which can be thought of as a rewarded Markov decision process with a single vertex.
The classification above is existential in nature and does not provide any insight on how a player guarantees a mean-payoff value. Our most technically challenging results concern the constructions strategies for Min and Max. The challenging part of the construction is reasoning about stronglyconnected bidding mean-payoff games. Consider a strongly-connected game in which Min can guarantee a non-positive mean-payoff value. The idea of our construction is to tie between changes in Min's budget with changes in the energy; investing one unit of budget (with the appropriate normalization) implies a decrease of a unit of energy, and on the other hand, an increase of a unit of energy implies a gain of one unit of budget. Since the budgets are bounded by 1, the value cannot increase arbitrarily. Finding the right bids in a general SCC is not trivial, and we find our solution to be surprisingly elegant. The case where Max can guarantee a positive mean-payoff value, is more challenging. Unlike a memoryless strategy for Min, the normalization factor must decrease as the value increases so that Max does not exhaust his budget. We show constant memory strategies in general and identify a fragment in which we show memoryless strategies.
Further bidding games Variants of bidding games where studied in the past. Already in [36] several variants are studied including a poorman version in which the winner of the bidding pays the bank, thus the amount of money in the game decreases as the game proceeds. Motivated by recreational games, e.g., bidding chess, discrete bidding games are studied in [23], where the money is divided into chips, so a bid cannot be arbitrarily small as in the bidding games we study. In all-pay bidding games [39], the players all pay their bids to the bank. Non-zero-sum two-player games were recently studied in [30]. They consider a bidding game on a directed acyclic graph. Moving the token throughout the graph is done by means of bidding. The game ends once the token reaches a sink, and each sink is labeled with a pair of payoffs for the two players that do not necessarily sum up to 0. They show existence of subgame perfect equilibrium for every initial budget and a polynomial algorithm to compute it.

Preliminaries
An arena is a pair G, α , where G is a directed graph and α is an objective. A game is played on an arena as follows. A token is placed on a vertex in the arena and the players move it throughout the graph. The outcome is an infinite path π. The winner or value is determined according to π and α as we elaborate below. There are several common modes in which the players move the token. In turn-based games the vertices are partitioned between the players and the player who controls the vertex on which the token is placed, moves it. Another mode is probabilistic choices, where the game can be thought of as a Markov chain, thus the edges are labeled with probabilities, and the edge on which the token proceeds is chosen randomly. A combination of these two modes is called 2.5player games, where the vertices are partitioned into three sets: Player 1 vertices, Player 2 vertices, and probabilistic vertices. Finally, in concurrent games, each player has a possible (typically finite) set of actions he can choose from in a vertex. The players select an action simultaneously, and the choice of actions dictates to which vertex the token moves.
We study a different mode of moving, which we call bidding. Both players have budgets, where for convenience, we have B 1 + B 2 = 1. In each turn, a bidding takes place to determine who moves the token. Both players submit bids simultaneously, where a bid is a real number in [0, B i ], for i ∈ {1, 2}. The player who bids higher pays the other player and decides where the token moves. Note that the sum of budgets always remains 1. While draws can occur, in the questions we study we try avoid the issue of draws.
A strategy prescribes to a player which action to take in a game, given a finite history of the game, where we define these two notions below. In 2.5-player games, histories are paths and actions are vertices. Thus, a strategy for Player i, for i ∈ {1, 2}, takes a finite path that ends in a Player i vertex, and prescribes to which vertex the token moves to next. In bidding games, histories and strategies are more complicated as they maintain the information about the bids and winners of the bids. A history is a sequence of the where, for j ≥ 1, in the j-th round, the token is placed on vertex v j−1 , the winning bid is b j , and the winner is Player i j , and Player i j moves the token to vertex v j . An action for a player is b, v ∈ ([0, 1] × V ), where b is the bid and v is the vertex to move to upon winning. An initial vertex v 0 and strategies f 1 and f 2 for Players 1 and 2, respectively, determine a unique outcome π for the game, denoted out(v 0 , f 1 , f 2 ), which is an infinite sequence in V · (V × [0, 1] × {1, 2}) ω . We sometimes abuse notation and refer to out(v 0 , f 1 , f 2 ) as a finite prefix of the infinite outcome. We drop v 0 when it is clear from the context. We define the outcome inductively. The first element of the outcome is v 0 . Suppose π 1 , . . . , π j is defined. The players bids are given by b 1 , v 1 = f 1 (π 1 , . . . , π j ) and b 2 , v 2 = f 2 (π 1 , . . . , π j ). If b 1 > b 2 , then π j+1 = v 1 , b 1 , 1 , and dually when b 1 < b 2 , we have π j+1 = v 2 , b 2 , 2 . We assume there is some tie-breaking mechanism that determines who the winner is when b 1 = b 2 , and our results are not affected by what the tie-breaking mechanism is. Consider a finite outcome π. The payment of Player 1 in π, denoted B 1 (π), is 1≤j≤|π| (−1) 3−ij b j , and Player 2's payment, denoted B 2 (π) is defined similarly. For i ∈ {1, 2}, consider an initial budget B init i ∈ [0, 1] for Player i. A strategy f is legal for Player i with respect to B init i if for every v 0 ∈ V and strategy g for the other player, Player i's bid in a finite outcome π = out(v 0 , f, g) does not surpass his budget. Thus, Richman games and threshold budgets The simplest qualitative objective is reachability: Player 1 has a target vertex v R and an infinite outcome is winning for him if it visits v R . Reachability bidding games are known as Richman games [37,36]. In Richman games both players have a target, which we denote by v R and v S . The game ends once one of the targets is reached. Note that this definition is slightly different from standard reachability games since there, Player 2 has no target and his goal is to keep the game from v R . Though, we show that for our purposes, since Richman games have no ties, reachability games are equivalent to Richman games (see Lemma 4).
The central question that is studied on bidding games regards a threshold budget. A threshold budget is a function THRESH : V → [0, 1] such that if Player 1's budget exceeds THRESH(v) at a vertex v, then he has a strategy to win the game. On the other hand, if Player 2's budget exceeds 1 − THRESH(v), he can win the game. We sometimes use THRESH 1 (v) to refer to THRESH(v) and THRESH 2 (v) to refer to 1 − THRESH(v). We formalize the problem of finding threshold budgets as a decision problem. We define the THRESH-BUDG problem, which takes as input a bidding game G, a vertex v, and a value t ∈ [0, 1], and the goal is to decide whether THRESH(v) = t.
Threshold values are shown to exist in [37] as well as how to compute them. We review briefly their results. Consider a Richman game G = V, E, v R , v S . We define the Richman function as follows. We first define R(v, i), for i ∈ N ∪ {0}, where the intuition is that if Player 1's budget exceeds R(v, i), he can win in at most i steps. We define R(v R , 0) = 0 and R(v, 0) = 1 for every other vertex v ∈ V . Indeed, Player 1 can win in 0 steps from v R no matter what his initial budget is, and even if he has all the budget, he cannot win in 0 steps from anywhere else. Consider i ∈ N and v ∈ V . We denote by adj(v) ⊆ V , the adjacent vertices to v, so u ∈ adj(v) iff E(v, u). Let v + be the vertex that maximizes the expression max u∈adj(v) R(u, i − 1), and let v − be the vertex that minimizes the expression min u∈adj(v) R(u, i − 1). We define R(v, i) = The following theorem shows that R(v) equals THRESH(v), and throughout the paper we use them interchangeably. We give the proof of the theorem for completeness. Proof. We prove for Player 1 and the proof for Player 2 is dual. Let t ∈ N be an index such We prove by induction on t that Player 1 wins in at most t steps. The base case is easy. For the inductive step, assume Player 1 has a budget of R(v, i) + . He bids . If he wins the bidding, he proceeds to v − with a budget of R(v − , i − 1) + . If he loses, then Player 2's bid exceeds b 1 and the worst he can do is move to v + . But then Player 1's budget is at least R(v + , i − 1) + . By the induction hypothesis, Player 1 wins in at most i − 1 steps from both positions.
We make precise the equivalence between reachability and Richman games.
Finding threshold budgets The authors in [36] study the complexity of threshold-budget problem and show that is in NP. They guess, for each vertex v its neighbors v − and v + , and devise a linear program with the constraints . The program has a solution iff the guess is correct. They leave open the problem of determining the exact complexity of finding the threshold budgets, and they explicitly state that it is not known whether the problem is in P or NP-hard.
We improve on their result by showing that THRESH-BUDG is in NP and coNP. Our reduction uses an important observation that is made in [37], which will be useful later on. They connect between threshold budgets and reachability probabilities in Markov chains.
We reduce THRESH-BUDG to the problem of "solving" a simple stochastic game (SSG, for short) [22]. An SSG has two players; one tries to minimize the probability that the target is reached, and the second player tries to minimize it. It is well-known that the game has a value, which is the probability of reaching the target when both players play optimally. The problem of finding the value of an SSG is known to be in NP ∩ coNP. The SSG we construct can be seen as a turn-based game in which the player whose turn it is to move is chosen uniformly at random.

Theorem 6. THRESH-BUDG for Richman games is in NP ∩ coNP.
Proof. We reduce the problem of finding the Richman value to the problem of finding the value in a 2.5-player reachability game, also called simple stochastic games [22] (SSGs, for short), which is known to be in NP and coNP. Let v R be Player 1's target vertex. Consider an SSG S and two strategies f and g for the two players. We can construct a Markov chain from S using f and g, and compute the probability that v R is reached when the two players follow f and g, which we denote by It is known that val(v) is achieved by memoryless strategies for both players.
Consider a Richman game G = V, E, v R , v S and a vertex v ∈ V . We construct an SSG S G by splitting every vertex v ∈ V , into a probabilistic vertex v c , a Player 1 vertex v 1 , and a Player 2 vertex v 2 . There are edges v c , v 1 and v c , v 2 with probability 1/2 each, which correspond to choosing randomly the player that moves next.
thus the claim follows, and we are done.
We stress the fact that the strategies in SSGs are very different from bidding games. As mentioned above, there, the strategies only prescribe which vertex to move the token to, whereas in bidding games, a strategy also prescribes what the next bid should be. So, a solution of a Richman game by reducing it to an SSG is existential in nature and does not give insight on the bids a player uses in his winning strategy. We will return to this point later on.
Objectives We study zero-sum games. The qualitative games we focus on are parity games. A parity game is a triple V, E, p , where p : V → {0, . . . , d} is a parity function that assigns to each vertex a parity index. An infinite outcome is winning for Player 1 iff the maximal index that is visited infinitely often is odd. The quantitative games we focus on are mean-payoff games. A mean-payoff game is V, E, w , where w : V → Z is a weight function on the vertices. We often refer to the sum of weights in a path as its energy. Consider an infinite outcome π = v 0 , v 1 , b 1 , i 1 , . . .. For n ≥ 0, we use π n to refer to the prefix of length n of π. The energy of π n , denoted E(π n ), is We define the mean-payoff value of π to be lim inf n→∞ n . The value of π can be thought of as the amount of money Player 1 pays Player 2. Note that the mean-payoff values do not affect the budgets of the players. That is, the game has two currencies: a "monopoly money" that is used to determine who moves the token and which the players do not care about once the game ends, and the mean-payoff value that is determined according to the weights of the vertices, which is the value that Min and Max seek to minimize and maximize, respectively. Consider a finite outcome π. We use B m (π) and B M (π) to denote the sum of payments of Min and Max in the bids. Throughout the paper we use m and M to refer to Min and Max, respectively.
Strategy complexity Recall that a winning strategy in a two-player game often corresponds to a system implementation. Thus, we often search for strategies that use limited or no memory. That is, we ask whether a player can win even with a memoryless strategy, which is a strategy in which the action depends only on the position of the game and not on the history. For example, in turn-based games, for i ∈ {1, 2}, a memoryless strategy for Player i prescribes, for each vertex v ∈ V i , a successor vertex u. It is well known that memoryless strategies are sufficient for winning in a wide variety of games, including turn-based parity games and turn-based mean-payoff games. In Richman games, the threshold budgets tell us who the winner of the game is. But, they do not give insight on how the game is won game, namely what are the bids the winning player bids in order to win. Particularly, when the threshold budgets are 0 as we shall see in Lemmas 7 and 15.
We extend the definition of memoryless strategies to bidding games, though the right definition is not immediate. One can define a memoryless strategy as a function from vertex and budget to action (i.e., bid and vertex) similar to the definition in other games. However, this definition does not preserve the philosophy of implementation with no additional memory. Indeed, recall the proof of Theorem 3. One can define a strategy that, given a vertex v ∈ V and a budget B, bids according to R t (v), where t is the minimal index such that R t (v) < B. Clearly, the memory that is needed to implement such a strategy is infinite.
To overcome this issue, we use a different definition. We define a memoryless strategy in a vertex v ∈ V with initial budget B ∈ [0, 1] as a pair u, f B v , where u ∈ adj(v) is the vertex to proceed to upon winning and f B v : [0, 1] → [0, 1] is a function that takes the current budget and, in mean-payoff games, also the energy, and returns a bid. We require that f B v is simple, namely a polynomial or a selection between a constant number of polynomials. For simplicity, we assume a memoryless strategy is generated for an initial vertex with an initial budget, thus there can be different strategies depending where the game starts and with what budget. Also, we call a concatenation of memoryless strategies, a memoryless strategy.

Parity Bidding Games
We study threshold budgets in bidding parity games. We first study strongly-connected parity games and show a classification for them; either Player 1 wins with every initial budget or Player 2 wins with every initial budget.

Lemma 7. Consider a strongly-connected parity game
Proof. The proof relies on the following claim: Player 1 wins a Richman game in which only his target is reachable, with every initial budget. The claim clearly implies the lemma as we view a strongly-connected bidding parity game as a Richman game in which Player 1 tries to force the game to the vertex with the highest parity index, and Player 2 has no target, thus Player 1 wins with every initial budget. The claim is similar for Player 2. The proof of the claim follows from the fact that the threshold budget of a vertex v ∈ V is some average between THRESH(v R ) and THRESH(v S ), and the average depends on the distances of v to the two targets. When only Player 1's target is reachable, we have THRESH(v) = 0.
We prove the claim for an odd maximal parity index, and the proof is similar for even maximal parity index. We claim that from every vertex v ∈ S, if Player 1 has a positive budget, he can force the game to reach the vertex u ∈ S with maximal parity index. Moreover, he can force the game to reach u with positive budget. A special vertex in S is u. Trivially, Player 1 can reach u, but we show that he can also reach u with positive budget in a non-trivial path, i.e., a loop.
We construct a Richman game on a graph V | S ∪ {t}, E , where t is a new vertex, and we redirect edges that have u as a target to the vertex t, thus for every e = v, w ∈ E| S , if w = u, then e ∈ E and otherwise v, t ∈ E . Player 1's objective is t and Player 2 has no objective (alternatively, his objective cannot be reached). We claim that R(v) = 0, for every v ∈ S. Assume towards contradiction that this is not the case. Let v be a vertex with maximal R(v) in S, and we denote γ = R(v). In particular, we have γ > 0. Since S is strongly connected, there is a path π from v to u. Let w be the last vertex on π before t.
Consider a vertex v ∈ S and assume Player 1's budget is 1 > 0. We describe a winning strategy for Player 1. By the above, Theorem 3, Player 1 has a strategy that forces the game to u. Furthermore, the winning strategy that is described there guarantees that u is reached with a positive budget. Let that budget be 2 > 0. Again, there is an index i 2 such that R(u, i 2 ) < 2 . We continue ad infinitum thereby guaranteeing that u is visited infinitely often. Since u has maximal parity index in S and it is odd, Player 1 wins the game.
Consider a bidding parity game G = V, E, p . Let R and S be the set of vertices in the BSCCs that are winning for Player 1 and Player 2, respectively. Let G be the Richman game that is obtained from G by setting the target of Player 1 to be the vertices in R and the target of Player 2 to be the vertices in S. The following lemma follows from Lemma 7.
Lemma 8 allows us to obtain the positive results of Richman games in parity bidding games. Theorem 9. The threshold budgets in parity bidding games exist, are unique, and THRESH-BUDG is in NP ∩ coNP.
We continue to study the strategy complexity in parity games. Proof. We show a memoryless strategy for Richman games. In order to obtain a memoryless strategy for parity games we proceed as the above; we first find a memoryless strategy in the Richman game in which the winning BSCCs are the targets for Player 1 and the losing BSCCs are the targets for Player 2, and then find a memoryless strategy in the internal Richman game by letting Player 1 draw the game to the vertex with maximal parity index. Consider a Richman game G = V, E, v R , v S , a vertex v 0 ∈ V , and an initial budget B init 1 ∈ [0, 1] for Player 1 with , we have > 0. We trim G by keeping only edges of the form v, v − , for every v ∈ V . Note that every vertex v that has R(v) < 1 has a path to v R . Let dist(v) be the distance from v R in the trimmed graph. For every vertex v ∈ V , there is a constant in [0, 1] that is sufficient for winning dist(v) times in a row. If Player 1's budget exceeds this constant, he bids accordingly and draws the game to v R . Otherwise, (v) . Note that if Player 1 wins for dist(v) times, he wins the game. Otherwise, he loses a bid and gains at least · 2 −|V | , thus eventually he will be able to win |V | times in a row.

Prompt winning
Motivated by the need to reason about the quality of systems, researchers have studied the question of prompt satisfaction of a specification [34]. Promptness in reachability games amounts to asking whether there is a bound k such that the target is reached by round k. Similarly, for Büchi games, we ask whether there is a bound k such that an accepting state is visited every k rounds (rather than the weaker property of visiting it infinitely often). We study promptness in bidding games, and we start with reachability games. The following theorem is an immediate corollary of the proof of Theorem 3. Theorem 11. Let t ∈ N be the minimal index such that B R > R(t, v), for some v ∈ V . Then, the reachability player can guarantee reaching v R in t rounds and the safety player can keep the game from reaching v R for t − 1 rounds.
We find our result for Büchi games somewhat surprising. Essentially, we show that the Büchi player cannot guarantee a prompt winning for every initial budget. Formally, we have the following.
Theorem 12. Consider a strongly-connected Büchi bidding game S = V, E and let F ⊆ V be a set of accepting vertices. If S contains a cycle C that does not traverse a vertex in F , then for every k ∈ N and every initial positive budget, Player 2 has a strategy that forces the game to cycle C for k times without visiting v. Thus, no accepting state is reached for at least k rounds.
Proof. We consider the "k-unwinding" G of S, which is G = V × {0, . . . , k}, E , where we describe E below. Intuitively, when the token is at level i, i.e., on a vertex in V × {i}, it means that C was traversed for i times without a visit to v. Accordingly, the edge that closes the cycle C leads from level i to level i + 1 and incoming edges to v lead from level i to level 0. Only Player 2 has a target, which are the vertices on the k-th level. Indeed, the k-th level is reached iff the cycle C is traversed k times. Since Player 2's goal is reachable from all vertices and Player 1 has no target, by Lemma 7, Player 2 can guarantee reaching for any positive initial budget, and we are done.

Mean-Payoff Bidding Games
We proceed to study mean-payoff games. We adjust the definition of threshold budgets to the quantitative setting.

Solving Bidding Mean-Payoff Games
In this section we solve the problem of finding threshold values in bidding mean-payoff games. Our solution relies on work on probabilistic models, namely one-counter simple stochastic games [14,13], and it is existential in nature. Namely, knowing what the threshold budget is in v does not give much insight on how Min guarantees a non-negative value even if he has sufficient budget, and similarly for Max. Constructing concrete memoryless strategies for the two players is much more challenging and we show constructions in the following sections.
Recall that in bidding parity games, we showed a classification for strongly-connected games; namely, the threshold budgets in all vertices are in {0, 1}, thus either Player 1 wins with every initial budget or Player 2 wins with every initial budget. We show a similar classification for stronglyconnected bidding mean-payoff games: the threshold budgets in all vertices of a strongly-connected bidding mean-payoff game are in {0, 1}, thus in a strongly-connected bidding mean-payoff game, for every initial energy and every initial budget, either Min can guarantee a non-positive mean-payoff value or Max can guarantee a positive mean-payoff value. The classification uses a generalization of the Richman function to weighted graphs. Consider a strongly-connected bidding mean-payoff game G = V, E, w and a vertex u ∈ V . We construct a graph G u = V u , E u , w u by making two copies u s and u t of u, where u s has no incoming edges and u t has no outgoing edges. Thus, a path from u s to u t in G u corresponds to a loop in G. Recall that we denote by w(v) the weight of the vertex v.

Definition 14. Consider a strongly-connected bidding mean-payoff game
We use the connection with probabilistic models as in Observation 5 in order to show that W is well defined. We view G u as a rewarded Markov chain, in which, for v ∈ V , the outgoing edges from v with positive probability probabilities are v, v + and v, v − , and their probability is 1/2. The function W coincides with the expected reward of a run that starts and returns to u, which in turn is well-defined since the probability of returning to u is 1.
Similarly to the connection we show in Theorem 6 between Richman values and reachability probabilities in a simple-stochastic game, we prove Lemma 15 by connecting the threshold value in bidding mean-payoff games to the probability that a counter in a one-counter simple-stochastic games reaches value 0. We then use results from [14,13] on this model to prove the lemma.

Lemma 15. Consider a strongly-connected bidding mean-payoff game
Proof. Consider a strongly-connected bidding mean-payoff game G = V, E, w . We construct a 2.5-player game S(G) similar to the proof of Theorem 6. We split every vertex v ∈ V into three vertices; a probability vertex v c , a Min vertex v m , and a Max vertex v M . We add edges with probability 1/2 from v c to v m and v M , and edges with probability 1 from v m and v M to every v c such that v, v ∈ E. The difference between the constructions is that now, the game has weights and we define the weight of v c to be w(v). The game we construct is referred to as one-counter SSG in [14]. There, the authors consider the objective of termination, i.e., reaching an energy level of 0.
Formally, for a vertex v ∈ V and an energy level i ∈ N, let T erm(v, i) be the set of infinite paths that start at v and have a decrease of at least i units of energy. It is shown in [14] that if there exists a vertex u ∈ V with W (u) ≥ 0 as well as a cycle from u to itself with negative sum of weights, then Min has a memoryless strategy f that has Pr f,g [T erm(v, i)] = 1, for every Max strategy g. On the other hand, by combining the results of [14] with [13], we have that if W (u) > 0, for all u ∈ V , then there exists a memoryless Max strategy g such that lim n→∞ Pr f,g [T erm(v, n)] = 0, for every Min strategy f .
In order to connect the threshold budget in G with the termination probability in S(G), we "unwind" both structures into a graph with infinite-many vertices in which we keep track of the accumulated energy. We refer to the unwinding of G as G ∞ . Intuitively, reaching a vertex v, i ∈ (V × N) in G ∞ corresponds to reaching v with energy i in G. The goal for Min is to reach energy level 0, thus G ∞ is a Richman game with infinite many vertices. The unwinding of S(G) coincides with G ∞ . Since G ∞ is locally finite, by [36], it has threshold budgets that satisfy the same properties as in finite graphs. When W (u) ≥ 0, we show that R( v, i ) = 0, for every v ∈ V and i ∈ N, thus Min can guarantee reaching an energy level of 0 from v with initial energy i, for every initial budget, thus by Lemma 16 he guarantees a non-negative mean-payoff value. On the other hand, if W (u) > 0, we can show that for every v ∈ V , we have lim i→∞ R( v, i ) = 1, thus for an initial Max budget B M , there is an initial energy level n such that 1 − R( v, n ) < B M . So, by Lemma 16, Max guarantees a positive mean-payoff value.
We show how to connect the Richman value of v, i in G ∞ with the termination probability in S(G). We prove for the case that there exists u ∈ V with W (u) ≥ 0, and the other case is similar. First note that if there are no negative-weight cycles in G, then there are also no positive-weight cycles, thus all outcomes have a mean-payoff value of 0. Suppose there is at least one negativeweight cycle in G. We construct from G an (unweighted) Richman game with infinite many vertices by keeping track of the accumulated energy throughout a play and setting Min's goal to reach energy Intuitively, reaching a vertex v, i ∈ V ∞ in G ∞ corresponds to reaching v with energy i in G. For convenience we assume the weights are on the edges rather than the vertices. Accordingly, for i, j ≥ 0, we have v, i , u, j ∈ E ∞ iff e = v, u ∈ E and w(e) = j−i, and v, i , v win ∈ E ∞ iff there is a vertex u ∈ V with e = v, u ∈ E and i + w(e) < 0. Note that G ∞ is locally finite. It is shown in [36] that threshold budgets exist for such infinite-graph games and that for every vertex Recall that a strategy for a player in a bidding game consists of two components; a bid and a vertex to move to upon winning. We fix the second component for Min: we call f m a Min strategy that, at a vertex v ∈ V , proceeds to v − ∈ V upon winning, where recall that v − is the neighbor of v with the minimal weighted-Richman value. We define a Richman value R f m for G ∞ , where Min proceeds according to f m . Using the same proof of [36], we can show that R f m is defined and for every . Note that f m can be seen as a Min strategy in S(G). Moreover, note that , where i and i are the appropriate energy levels. Since memoryless strategies suffice in one-counter SSGs, there is a Max memoryless strategy f M for which Assume towards contradiction that Remark. In the following sections we show an alternative direct proof to Lemma 15 (see Theor-ems 22 and 33), which does not use the probabilistic connection and constructs concrete strategies for the players. We include both proofs for Lemma 15 since we find the probabilistic connection important.
Lemma 16, which is also helpful in the following sections, shows how to connect the meanpayoff value with the objective of reaching energy 0 or maintaining non-negative energy.
Lemma 16. Consider a strongly-connected bidding mean-payoff game G and a vertex u in G. Suppose that for every initial budget and initial energy, Min has a strategy f m and there is a constant N ∈ N such that for every Max strategy f M , a finite outcome π = out(u, f m , f M ) either reaches energy 0 or the energy is bounded by N throughout π. Then, Min can guarantee a non-positive mean-payoff value in G. If for every initial budget B init M ∈ [0, 1] for Max there exists an initial energy level n ∈ N such that Max can guarantee a non-negative energy level in G, then Max can guarantee a positive mean-payoff value in G.

Proof.
We start with the first claim. Suppose Min has a strategy f m as the above, and we describe a Min strategy f m that guarantees a non-positive mean-payoff value. Min plays according to f m until an energy of 0 is reached. He bids 0 until the energy increases, then he forces the game back to u, which is possible due to Lemma 7, and plays again according to f m . Since the mean-payoff value of an infinite outcome is the lim inf of the mean-payoff values of its finite prefixes, reaching 0 energy infinitely often implies a non-positive mean-payoff value. On the other hand, consider an infinite outcome π 1 · π 2 ∈ out(u, f m , f M ), where f M is some Max strategy and π 1 is the prefix after which 0 energy is never reached. Then, the energy level in π 2 is bounded by N . Thus, the mean-payoff value of π is 0, and we are done.
We proceed to the second claim. Consider an initial budget B init M for Max. Let = W (v) · 2 v∈V cont(v) −1 and let G be the game obtained from G by decreasing all the weights by .
Then, the weighted Richman value of u in G is W (u)/2 and in particular, it is positive. Thus, there exists n ∈ N such that Max can keep the energy level in G non-negative with an initial budget of B init M , if the initial energy level is n. Max plays in G according to his strategy in G . Thus, he guarantees that for every finite outcome π in G, the energy is at least |π| − n. Since n is constant, the mean-payoff value of an infinite outcome is at least .
Deciding the classification in Lemma 15 can be done in NP and coNP by guessing the neighbors the vertices and using linear programming, similarly to Richman games. Then, we reduce bidding mean-payoff games to Richman games in a similar way to the proof of Lemma 8 for parity games.
Theorem 17. Threshold budgets exist in bidding mean-payoff games, they are unique, and THRESH-BUDG for bidding mean-payoff games is in NP ∩ coNP.
Proof. Consider a bidding mean-payoff game G = V, E, w . Lemma 15 induces a reduction from bidding mean-payoff games to Richman games: we classify the BSCCs of G, relate Min with Player 1 and Max with Player 2, and set the BSCCs with threshold budget 0 as Player 1's target and these with threshold budget 1 as Player 2's target. For every v ∈ V that is not in a BSCC, we have THRESH(v) = R(v). Indeed, if Min's budget exceeds R(v) in v, he can draw the game to a BSCC from which he can guarantee a non-negative mean-payoff value, and the claim for a budget below R(v) is dual. Thus, we only need to show how to determine whether a BSCC S of G has a vertex u ∈ S with W (u) ≤ 0. This can easily be done in NP and coNP. For a vertex u ∈ S we guess, for every v ∈ S two neighbors v + and v − , and add the appropriate linear constraints: we have W (u t ) = 0, for every neighbor . Such a system can be solved in polynomial time, thus we are done.

A Memoryless Optimal Strategy for Min
We turn to the more challenging task of finding memoryless strategies for the players, and in this section we focus on constructing a strategy for Min. Theorem 10 and Lemma 15 allow us to focus on strongly-connected bidding mean-payoff games. Consider a strongly-connected bidding meanpayoff game G = V, E, w that has a vertex u ∈ V with W (u) ≤ 0. We construct a Min memoryless strategy that guarantees that for every initial energy and every initial budget, either the energy level reaches 0 or it is bounded. By Lemma 16, this suffices for Min to guarantee a non-positive mean-payoff value in G.
Constructing a memoryless strategy in strongly-connected bidding mean-payoff games is much more challenging than strongly-connected bidding parity games. The idea behind our construction is to tie between changes in the energy level and changes of the budget. That is, in order to decrease the energy by one unit, Min needs to invest at most one unit of budget (with an appropriate normalization), and when Max increases the energy by one unit, Min's gain is at least one unit of budget. Our solution builds on an alternative solution to the two-loop game in Figure 1. This solution is inspired by a similar solution in [36]. Indeed, given the result in Theorem 12, one can suspect that Max can force the game to arbitrarily high value. The idea behind our solution is to prevent such an outcome by tying between changes in the energy level and changes of the budget.

Example 18.
Consider the bidding mean-payoff game that is depicted in Figure 1. We show a Min strategy that guarantees a non-positive mean-payoff value. Consider an initial Min budget of B init m ∈ [0, 1] and an initial energy level of k I ∈ N. Let N ∈ N be such that B init m > k N . Min bids 1 N and takes the (−1)-weighted edge upon winning. Intuitively, Min invests 1 N for every decrease of unit of energy and, since by losing a bidding he gains at least 1 N , this is also the amount he gains when the energy increases. Formally, it is not hard to show that the following invariant is maintained: if the energy level reaches k ∈ N, Min's budget is at least k N . Note that the invariant implies that either an energy level of 0 is reached infinitely often, or the energy is bounded by N . Indeed, in order to cross an energy of N , Max would need to invest a budget of more than 1. Lemma 16 implies that the mean-payoff value is non-positive, and we are done.
Extending this result to general strongly connected games is not immediate. Consider a stronglyconnected game G = V, E, w and a vertex u ∈ V . We would like to maintain the invariant that upon reaching u with energy k, the budget of Min exceeds k/N , for a carefully chosen N . The game in the simple example above has two favorable properties that general SCCs do not necessarily have. First, unlike the game in the example, there can be infinite paths that avoid u, thus Min might need to invest budget in drawing the game back to u. Moreover, different paths from u to itself may have different energy levels, so bidding a uniform value (like the 1 N above) is not possible. The solution to these problems is surprisingly elegant and uses the weighted Richman function in Definition 14.
Consider an initial budget of B init m ∈ [0, 1] for Min and an initial energy k I ∈ N. We describe · 1 N and he proceeds to v − upon winning, where we choose N ∈ N in the following. Let w M be the maximal absolute weighted In the following lemmas we prove that f m guarantees that an outcome either reaches energy level 0 or that the energy is bounded, as well as showing that f m is legal, i.e., that Min always bids less than his budget. The following lemma is the crux of the construction as it connects the weighted Richman function with the change in energy and in budget. Recall that for a finite outcome π the accumulated energy in π is E(π) and the payments of Min throughout π is B m (π).

Lemma 19.
Consider a Max strategy f M , and let π = out(f m , f M ) be a finite outcome that starts in a vertex v and ends in v . Then, we have W (v) − W (v ) ≥ E(π) + N · B m (π).

XX:15
Proof. We prove by induction on the length of π. In the base case v = v , thus E(π) = B m (π) = 0 and the claim is trivial. For the induction step, let b be the winning bid in the first round and let π be the suffix of π after the first bidding. We distinguish between two cases. In the first case, Min wins the bidding, N , and proceeds to v − . Thus, we have E(π) + N · B m (π) = w(v) + E(π ) + N b + B m (π ) . By the induction hypothesis, we have , and we are done.
The following corollary of Lemma 19 explains why we refer to our technique as "tying energy and budget". Its proof follows from the fact that W (u s ) ≤ 0 and W (u t ) = 0.

Corollary 20.
Consider a Max strategy f M , and let π = out(f m , f M ) be a finite outcome from u to u. Then, we have −N · B m (π) ≤ E(π).
We formalize the intuition above by means of an invariant that is maintained throughout the outcome. Recall that the game starts from a vertex u ∈ V with W (u) ≤ 0, the initial energy is

Lemma 21.
Consider a Max strategy f M , and let π = out(f m , f M ) be a finite outcome. Then, when the energy level reaches k, Min's budget is at least k+b M N .
Proof. The invariant clearly holds initially. Consider a partition π = π 1 · π 2 , where π 1 is a maximal prefix of π that ends in u and π 2 starts in u and ends in a vertex v ∈ V . The energy level at the end of π is k = k I + E(π). Recall that B m (π) is the sum of Min's payments in π, thus his budget at the end of π is B init m − B m (π 1 ) + B m (π 2 ) . By Corollary 20, we have −B m (π 1 ) ≥ 1 N E(π 1 ) and by Lemma 19,we  Lemma 21 implies that Min always has sufficient budget to bid according to f m , thus the strategy is legal. Moreover, since Min's budget cannot exceed 1, Lemma 21 implies that if the energy does not reach 0, then it is bounded by N − b M . Thus, Lemma 16 implies that Min has a memoryless strategy that guarantees a non-positive mean-payoff value in a strongly-connected bidding mean-payoff game having a vertex u with W (u) ≤ 0. Combining with the memoryless strategy in parity games, we have the following.

Theorem 22.
Consider a bidding mean-payoff game G = V, E, w and a vertex v ∈ V . If Min's initial budget exceeds THRESH(v), he has a memoryless strategy that guarantees a nonpositive mean-payoff value.

A Memoryless Optimal Strategy for Max
The complementary result of the previous section is more involved. Consider a strongly-connected bidding mean-payoff game G with a vertex u that has W (u) > 0. We devise a Max strategy that guarantees a positive mean-payoff value in G. We start with a fragment of the general case called recurrent SCCs, and we generalize our solution later. We say that an SCC G = V, E is a recurrent, if there is a vertex u ∈ V such that every cycle in G includes u. We refer to u as the root of G.
Intuitively, the construction has two ingredients. First, we develop the idea of tying energy and budget. We construct a Max strategy f M that guarantees the following: when Max invests a unit of budget (with an appropriate normalization), then the energy increases by at least one unit, and when the energy decreases by one unit, Max's gain is at least z > 1 units of budget, where z arises from the game graph. The second ingredient concerns the normalization factor. Recall that in the previous section it was a constant 1 N . Here on the other hand, it cannot be constant. Indeed, if the normalization does not decrease as the energy increases, Max's budget will eventually run out, which is problematic since with a budget of 1, Min can guarantee reaching energy level 0, no matter how high the energy is. The challenge is to decide when and how to decrease the normalization factor. We split N into energy blocks of size M , for a carefully chosen M ∈ N. The normalization factor of the bids depends on the block in which the energy is in, and we refer to it as the currency of the block. The currency of the n-th block is z −n . Note that the currency of the (n − 1)-th block is higher by a factor of z from the currency of the n-th block. This is where the first ingredient comes in: investing in the n-th block is done in the currency of the n-th block, whereas gaining in the n-th block is in the higher currency of the (n − 1)-th block. We switch between the currencies when the energy moves between energy blocks only at the root u of G. This is possible since G is a recurrent SCC. The mismatch between gaining and investing is handy when switching between currencies as we cannot guarantee that when we reach u the energy is exactly in the boundary of an energy block.
We formalize this intuition. We start by finding an alternative definition for the weighted Richman function. Recall that in order to define W , we constructed a new graph G u by splitting u into u s and u t . We define the contribution of a vertex v ∈ V to W (u s ), denoted cont(v), as follows. We

Lemma 23.
Consider a strongly-connected mean-payoff game G = V, E, w , which is not necessarily recurrent. We have W (u) = v∈V cont(v) · w(v) .
Proof. The proof uses the connection with probabilistic models, and follows from standard arguments there. Let M be a rewarded Markov chain on the structure of G, where edges of the form v, v + and v, v − have probability 1/2 each and all other edges have probability 0. As observed above, W (u) coincides with the expected reward of a path from u to itself in M, which by standard probabilistic arguments, coincides with the long-run average reward in M. In turn, the long-run average reward equals v∈V µ(v) · w(v), where µ(v) is the invariant distribution of v, which is exactly cont(v) in G. Thus, the claim follows.
we have z > 1. Let G z be the game that is obtained from G by multiplying the negative-weighted vertices by z, We denote by W z the weighted threshold budgets in G z . The following lemma follows immediately from Lemma 23.
We define the partition into energy blocks. Let cycles(u) be the set of simple cycles from u to itself and w M = max π∈cycles(u) |E(π)|. We choose M such that where b M is the maximal bid as in the previous section. We partition N into blocks of size M . For n ≥ 1, we refer to the n-th block as M n , and we have M n = {M (n−1), M (n−1)+1, . . . , M n−1}. We use β ↓ n and β ↑ n to mark the upper and lower boundaries of M n , respectively. We use a M ≥n to denote the set {M n , M n+1 , . . .}. Consider a finite outcome π that ends in u and let visit u (π) be the set of indices in which π visits u. Let k I ∈ N be an initial energy. We say that π visits M n Min invests a budget of more than 1. Lemma 16 implies that Max guarantees a positive mean-payoff value in a strongly-connected game.

An optimal Max strategy in general games
In this section we develop a constant-memory strategy for Max that guarantees a positive meanpayoff value. The difficulty lies in coping with outcomes in which the energy forms a sine-like wave on the boundary of an energy block. In recurrent SCCs, we can change currency every time the wave changes block, which does not work in general strongly-connected games as we show in the following example. Example 29. Consider the strongly-connected mean-payoff game G that is depicted in Figure 3. Note that there can be an infinite outcome that stays in v 1 . Moreover note that since v + 1 = v 1 , upon winning a bidding in v 1 , Max stays in v 1 . Thus, currency changes must occur in v 1 as otherwise Max's budget will eventually run out. The strategy in the previous section changes currency whenever the energy changes blocks. We show that naively changing currency in v 1 whenever a change in energy block occurs, is also not possible. Note that in this game, using the definition of the previous section, we have z = 2. Suppose the game starts in u in the third energy block, thus the normalization factor is 1/8. Consider the outcome in which Max wins with b z = 3/8 in u and moves to v 1 , wins again with b z = 1/2 · 1/8 and stays in v 1 , and then loses twice, thus the gain is at least 1/8(1/2 + 4) and the game returns to u. The change in energy is 0.5 + 0.5 + −1 = 0. Now, suppose the energy at u is at the top of the third block. So, after visiting v 1 twice, the energy moves to the fourth block and the currency changes to 1/16. Thus, Max pays the first two wins in the high currency: 3 · 1/8 + 0.5 · 1/8 and gains in the low currency: −0.5 · 1/16 − 4 · 1/16. All in all, Max's payments are positive, thus his budget decreases, while the energy level stays the same. Min can perform bids so that this outcome repeats, thus eventually exhausting Max's budget.
We develop further the two ingredients that are used in the previous section. First, recall that investing in an energy block M n is in the currency of the n-th block, whereas gaining is in the higher currency of the (n − 1)-th block. In general games, we need a stronger property; investing in M n is in the lower currency of the (n + 1)-th block while gaining is still in the higher currency of the (n − 1)-th block. Next, we differentiate between even blocks, i.e., M 2n , and odd blocks, i.e., M 2n+1 , for some n ∈ N. When the energy level reaches an even block M 2n , the currency used is z −n . In order to determine the currency in the odd blocks, we take the history of the play into account; the currency matches the currency in the last energy block that was visited before entering M 2n+1 . Hence, we call our strategy a constant-memory strategy. The odd blocks serve as "buffers" so that when we change currency, there is a sufficiently large change in energy that in turn implies that Max's budget sufficiently increases compared with the change in energy. Combining with the memoryless strategy in parity games of Theorem 10, we have the following.
We formalize this intuition. Consider a strongly-connected mean-payoff bidding games Since W (u) > 0, we have z > 1. We define a new weight functionw z : V → IR as follows LetG z = V, E,w z andW z be the weighted-Richman values inG z . For a finite outcome π, letẼ z be the sum of weights that π traverses inG z . The first part of following lemma is immediate from the definition ofG z and the second is proven similarly to Lemma 26.

Lemma 30.
The following two properties hold. W z (u) = 0. Consider a path π. We haveẼ z (π) ≤ z · E(π) andẼ z (π) ≤ 1 z · E(π). We construct a Max strategy f M that uses constant memory. When reaching a vertex v ∈ V , Max bids 1 2 W z (v + ) −W z (v − ) · γ and moves to v + upon winning, where γ ∈ (0, 1) is the normalization factor, which recall that we refer to as the currency. Unlike the previous section in which the currency changes occur only in the root, here, changes can occur anywhere. In order to choose the currency, similarly to the previous section, we partition N into blocks ofM ∈ N, where we chooseM later on. We refer to the n-th block asM n = {M · (n − 1), . . . ,M n − 1}. We distinguish between even blocks, namelyM 2n , and odd blocks, namelyM 2n+1 , for n ∈ N. When the energy level reaches an even blockM 2n , the currency is z −n . In order to determine the currency in the odd blocks, we take the history of the play into account; the currency matches the currency in the last energy block that was visited before enteringM 2n+1 . Thus, if it isM 2n , then the currency is z −n and otherwise, it is the energy blockM 2n+2 , and the currency is z −(n+1) . We say that a finite outcome is γ-consistent when all the bids Max performs in it are made in the same currency γ. Lemma 25 clearly follows to γ-consistent outcomes. Let w M = max v∈V |W (v)|. The following lemma follows from combining Lemma 25 with Lemma 30.

Lemma 31.
Consider a (z −n )-consistent outcome π that starts in v and ends in v . We have Suppose Max is playing according to f M and Min is playing according to some strategy f m . Let π = out(f m , f M ) be the infinite outcome. Let π = π 1 ·π 2 ·. . . be a partition of π into maximal finite outcomes that have a consistent currency. As in the previous section, for i ≥ 1, let i ∈ N be the energy at the end of π i , thus i = k I + E(π 1 . . . π i ), where k I is an initial energy. Also, let β ↑ n and β ↓ n be respectively, the upper and lower boundaries of the energy block M n . Note that β ↑ n = β ↓ n+1 . Suppose a subpath π i starts in a vertex v and ends in v . We make observations on the budget change during π i . There are four cases, which are depicted in Figure 4. Note that the currency in Cases 1 and 3 is z −n and in Cases 2 and 4 it is z −(n+1) . The energy change in π i in Cases 1 and 2 is at least 2M and at most 2M + 2w M and in Cases 3 and 4 it is at least M and at most M + 2w M . We use Lemma 31 to obtain the following: Lemma 32. The following bounds hold for the change in budget in the four cases depicted in Figure 4.
To conclude the construction, given an initial Max budget, we find an initial energy level k I with which Max can guarantee winning. We do this by finding an invariant on his budget at the end points of energy blocks. Recall the intuition that increasing the energy within an energy block costs one unit of budget in the currency of the block. Thus, similarly to the previous section, the cost of energy blockM 2n , for n ∈ IN, isM · z −n . The difficulty is in chargingM 2n+1 since the currency depends on whether the energy increases or decreases. We show that charging with the higher currency works, thus the cost ofM 2n+1 is alsoM · z −n .
Recall from the previous section that Max's budget at the bottom of an energy blockM 2n , needs to include, in addition to the costs of the energy blocksM ≥2n , wiggle room in the currency of the lower block. Going back to the Lemma 32, we observe that Case 4 is the only problematic case. Indeed, in all other cases, the path π i crosses an energy block whose cost is given in a currency that is lower than the currency of gaining (when decreasing), or higher than the currency of investing (when increasing). Take for example Case 2. It crosses bothM 2n+2 andM 2n+1 . The cost ofM 2n+2 isM · z −(n+1) whereas the gain for it is roughlyM · z −n . The situation in Case 4 is not that bad. The gain equals the cost ofM 2n+1 , i.e.,M · z −n , up to a constant, i.e., 2w M · z −n . We add this constant in the invariant, thus we require Max's budget at β ↑ 2n+1 to include the costs of the higher blocks, the wiggle room, and a surplus of 2w M · z −n .
We define the invariant on Max's budget formally. Recall that wiggle = 2w M + b M , where b M is the maximal bid, and it is used to guarantee that f M is legal in an outcome that stays in an energy block. We write Inv(β ↑ ) to refer to the budget that Max has when the currency changes near β ↑ , thus within |w M | of β ↑ . We have the following.
To conclude the construction, we chooseM to be large enough so that the invariant is maintained assuming it is maintained initially. Also, given an initial budget for Max, we choose an initial energy level such that the invariant is initially maintained. We thus have the following. Theorem 33. Consider a bidding mean-payoff game G = V, E, w . For a vertex v ∈ V , if Max has an initial budget that is greater than 1 − THRESH(v), he has a constant-memoryless strategy that guarantees a positive mean-payoff value.

Discussion and Future Directions
We introduce and study infinite-duration bidding games in which the players bid for the right to move the token. This work belongs to a line of works that transfer concepts and ideas between the areas of formal methods and algorithmic game theory (AGT, for short). A core field of AGT is the study of auctions, and the bidding in a bidding game can be seen as a simple auction for determining which player moves. Richman games originated in the game theory community in the 90s and recently gained interest by the AGT community [30]. We combine them with the study of infinite-duration games, which is well-studied in the formal methods community. Prior to this work, a series of works focused on applying concepts and ideas from formal methods to resource-allocation games [10,8,9,6,7,35], which constitutes a well-studied class of games in AGT. More to the formal methods side, there are many works on games that share similar concepts to these that are studied in AGT. For example, logics for reasoning about multi-agent systems [3, 19,40], studies of equilibria in games related to synthesis and repair problems [18,25,1,15], and studies of infinite-duration non-zero-sum games [21,16,17,12].
There are several problems we left open as well as plenty of future research directions. We list a handful of them below. We showed that the complexity of THRESH-BUDG is in NP and coNP. We leave open the problem of determining its exact complexity. We conjecture that it is reducible from solving simple stochastic games, which will show that it is as hard as several other problems whose exact complexity is unknown. In this work we focused on parity and mean-payoff games. Energy games are games that are played on a weighted graph, where one of the players tries to reach negative energy and the second player tries to prevent it. Note that unlike parity and mean-payoff, the energy objective is not prefix independent. We can show that threshold budgets exist in energy games. The complexity of THRESH-BUDG in energy games is interesting and is tied with recent work on optimizing the probability of reaching a destination in a weighted MDP [26,42]. For acyclic energy bidding games, the problem is PP-hard using a result in [26], and for a single-vertex games the problem is in P using the direct formula of [32]. For general games the problem is open.