Document Open Access Logo

Asymmetrically Discounted Stochastic Games

Authors Sarvin Bahmani , Soumyajit Paul , Sven Schewe , Shadi Tasdighi Kalat , Ashutosh Trivedi

PDF

File

Thumbnail PDF
PDF
LIPIcs.CONCUR.2026.14.pdf
  • Filesize: 1.44 MB
  • 22 pages

Document Identifiers

Subject Classification

ACM Subject Classification
  • Mathematics of computing → Markov processes
  • Theory of computation → Convergence and learning in games
Keywords
  • Stochastic Games
  • Asymmetric Discounting
  • Stackelberg Equilibrium

Metrics

Abstract

We study asymmetrically discounted stochastic games, in which players use distinct and reasonably apart discount factors. We show that optimal strategies in these games may require both memory and randomisation, in contrast to the classical symmetrically discounted setting. Our main technical contribution establishes that computing incentive Stackelberg equilibria - a variant of Stackelberg equilibria in which one player, called Player Max, can offer payments to the other player, called Player Min - is no harder than solving classical discounted games. We further show that optimal strategies in this setting can be realised by finite counting strategies, whereas restricting players to stationary strategies makes the problem computationally intractable. Finally, we establish that computing classical Stackelberg equilibria in these games under the constraint of memoryless strategies is NP-complete and remains NP-hard even when general or counting strategies are allowed.

Cite As Get BibTex

Sarvin Bahmani, Soumyajit Paul, Sven Schewe, Shadi Tasdighi Kalat, and Ashutosh Trivedi. Asymmetrically Discounted Stochastic Games. In 37th International Conference on Concurrency Theory (CONCUR 2026). Leibniz International Proceedings in Informatics (LIPIcs), Volume 391, pp. 14:1-14:22, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2026) https://doi.org/10.4230/LIPIcs.CONCUR.2026.14

Author Details

Sarvin Bahmani
  • University of Liverpool, UK
Soumyajit Paul
  • University of Liverpool, UK
Sven Schewe
  • University of Liverpool, UK
Shadi Tasdighi Kalat
  • University of Colorado Boulder, CO, USA
Ashutosh Trivedi
  • University of Colorado Boulder, CO, USA
  • University of Liverpool, UK

Funding

This work was supported by the EPSRC through grants EP/X03688X/1 (TRUSTED) and EP/X042596/1 (Games for Good), and in part by the NSF under CAREER Award CCF-2146563. Ashutosh Trivedi is a Royal Society Wolfson Visiting Fellow and gratefully acknowledges the support of the Wolfson Foundation and the Royal Society.

Supplementary Materials

References

  1. M. Agranov, J. Kim, and L. Yariv. Coordination with differential time preferences: Experimental evidence. American Economic Review: Insights, pages 543-57, December 2024. Google Scholar
  2. S. Bahmani, S. Paul, S. Schewe, S. Tasdighi Kalat, and A. Trivedi. Experiments for asymmetrically-discounted stochastic games, June 2026. URL: https://doi.org/10.5281/zenodo.21101820.
  3. L. C. Baird. Advantage updating, 1993. Google Scholar
  4. Moshe Bar. Wait for the second marshmallow? future-oriented thinking and delayed reward discounting in the brain. Neuron, pages 4-5, 2010. Google Scholar
  5. K. Chatterjee and R. Ibsen-Jensen. Strategy complexity of finite-horizon Markov decision processes and simple stochastic games. In Proc. of MEMICS, pages 106-117, 2012. Google Scholar
  6. K. Chatterjee, J.-P. Katoen, M. Weininger, and T. Winkler. Stochastic games with lexicographic reachability-safety objectives. In Proc. of CAV, LNCS, pages 398-420, 2020. Google Scholar
  7. K. Chatterjee and Y. Velner. Mean-payoff pushdown games. In Proc. of LICS, 2012. Google Scholar
  8. B. Chen and S. Takahashi. A folk theorem for repeated games with unequal discounting. Games and Economic Behavior, pages 571-581, 2012. Google Scholar
  9. V. Conitzer and T. Sandholm. Computing the optimal strategy to commit to. In Proc. of the 7th ACM conference on Electronic commerce, pages 82-90, 2006. Google Scholar
  10. A. Dasgupta and S. Ghosh. Self-accessibility and repeated games with asymmetric discounting. Journal of Economic Theory, page 105312, 2022. Google Scholar
  11. Jerzy A Filar and Koos Vrieze. Competitive Markov decision processes. Springer Science & Business Media, 1997. Google Scholar
  12. Y. Guéron, T. Lamadon, and C. D Thomas. On the folk theorem with one-dimensional payoffs and different discount factors. Games and Economic Behavior, pages 287-295, 2011. Google Scholar
  13. A. Gupta and S. Schewe. It pays to pay in bi-matrix games-a rational explanation for bribery. Proc. of IFAAMAS, 2015. Google Scholar
  14. A. Gupta and S. Schewe. Buying optimal payoffs in bi-matrix games. Games, page 40, 2018. URL: https://doi.org/10.3390/G9030040.
  15. A. Gupta, S. Schewe, A. Trivedi, M. S. K. Deepak, and B. K. Padarthi. Incentive Stackelberg mean-payoff games. In Proc. of SEFM, pages 304-320, 2016. Google Scholar
  16. T. D. Hansen, P. B. Miltersen, and U. Zwick. Strategy iteration is strongly polynomial for 2-player turn-based stochastic games with a constant discount factor. J. ACM, pages 1:1-1:16, 2013. URL: https://doi.org/10.1145/2432622.2432623.
  17. Marcin Jurdzinski. Deciding the winner in parity games is in UP ∩ co-UP. Inf. Process. Lett., pages 119-124, 1998. Google Scholar
  18. Olav Kallenberg. Foundations of modern probability. Springer, 1997. Google Scholar
  19. M. Kwiatkowska, G. Norman, D. Parker, and C. Wiltsche. Prism-games 3.0: Stochastic game verification with concurrency, equilibria and time. In Proc. of CAV, pages 485-497, 2020. Google Scholar
  20. E. Lehrer and A. Pauzner. Repeated games with differential time preferences. Econometrica, pages 393-412, 1999. Google Scholar
  21. Fink A M. Equilibrium in a stochastic n-person game. Hiroshima Math J, 28:89-93, 1964. Google Scholar
  22. J.-F. Mertens and A. Neyman. Stochastic games. Int. J. Game Theory, pages 53-66, 1981. Google Scholar
  23. Walter Mischel. The marshmallow test: Understanding self-control and how to master it. Random House, 2014. Google Scholar
  24. M. L. Puterman. Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley & Sons, Inc., New York, NY, USA, 1994. Google Scholar
  25. A. Rubinstein. Perfect equilibrium in a bargaining model. Econometrica, pages 97-109, 1982. Google Scholar
  26. L. S. Shapley. Stochastic games. Proc. Nat. Acad. Sci. U.S.A., pages 1095-1100, 1953. Google Scholar
  27. R. S. Sutton and A. G. Barto. Reinforcement Learning: An Introduction. MIT Press, 2nd edition, 2018. Google Scholar
  28. S. Tasdighi Kalat, S. Sankaranarayanan, and A. Trivedi. What is your discount factor? In International Conference on Quantitative Evaluation of Systems and Formal Modeling and Analysis of Timed Systems, pages 322-336, 2024. Google Scholar
  29. Heinrich von Stackelberg. Marktform und Gleichgewicht. Springer, Vienna, 1934. Google Scholar
Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2026 Schloss Dagstuhl – LZI GmbH Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact