Document Open Access Logo

Finding Equilibria: Simpler for Pessimists, Simplest for Optimists

Authors Léonard Brice , Thomas A. Henzinger , K. S. Thejaswini

PDF HTML 5

Files

Thumbnail PDF
PDF
LIPIcs.MFCS.2025.30.pdf
  • Filesize: 1.09 MB
  • 18 pages
HTML5 Logo HTML (experimental)
LIPIcs.MFCS.2025.30.html

Document Identifiers

Related Versions

Subject Classification

ACM Subject Classification
  • Theory of computation → Models of computation
Keywords
  • Nash equilibria
  • stochastic games
  • graph games
  • risk-sensitive equilibria

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Abstract

We consider equilibria in multiplayer stochastic graph games with terminal-node rewards. In such games, Nash equilibria are defined assuming that each player seeks to maximise their expected payoff, ignoring their aversion or tolerance to risk. We therefore study risk-sensitive equilibria (RSEs), where the expected payoff is replaced by a risk measure. A classical risk measure in the literature is the entropic risk measure, where each player has a real valued parameter capturing their risk-averseness. We introduce the extreme risk measure, which corresponds to extreme cases of entropic risk measure, where players are either extreme optimists or extreme pessimists. Under extreme risk measure, every player is an extremist: an extreme optimist perceives their reward as the maximum payoff that can be achieved with positive probability, while an extreme pessimist expects the minimum payoff achievable with positive probability. We argue that the extreme risk measure, especially in multi-player graph based settings, is particularly relevant as they can model several real life instances such as interactions between secure systems and potential security threats, or distributed controls for safety critical systems. We prove that RSEs defined with the extreme risk measure are guaranteed to exist when all rewards are non-negative. Furthermore, we prove that the problem of deciding whether a given game contains an RSE that generates risk measures within specified intervals is decidable and NP-complete for our extreme risk measure, and even PTIME-complete when all players are extreme optimists, while that same problem is undecidable using the entropic risk measure or even the classical expected payoff. This establishes, to our knowledge, the first decidable fragment for equilibria in simple stochastic games without restrictions on strategy types or number of players.

Cite As Get BibTex

Léonard Brice, Thomas A. Henzinger, and K. S. Thejaswini. Finding Equilibria: Simpler for Pessimists, Simplest for Optimists. In 50th International Symposium on Mathematical Foundations of Computer Science (MFCS 2025). Leibniz International Proceedings in Informatics (LIPIcs), Volume 345, pp. 30:1-30:18, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2025) https://doi.org/10.4230/LIPIcs.MFCS.2025.30

Author Details

Léonard Brice
  • Université Libre de Bruxelles, Belgium
Thomas A. Henzinger
  • Institute of Science and Technology Austria, Klosterneuburg, Austria
K. S. Thejaswini
  • Institute of Science and Technology Austria, Klosterneuburg, Austria

Funding

This work is a part of project VAMOS that has received funding from the European Research Council (ERC), grant agreement No 101020093.

Acknowledgements

We thank anonymous reviewers for pointing us to the Hurwicz criterion and to the work of Gallego-Hernández and Mansutti [Jorge Gallego-Hernández and Alessio Mansutti, 2025]. We thank Marie van den Bogaard for her valuable feedback on the first author’s PhD dissertation, which helped improve the quality of this work.

References

  1. Agarwal Alekh, Nan Jiang, Sham M. Kakade, and Sun Wen. Reinforcement Learning: Theory and Algorithms. https://rltheorybook.github.io/, 2021. Google Scholar
  2. M. Auer. Hands-On Value-at-Risk and Expected Shortfall: A Practical Primer. Management for Professionals. Springer International Publishing, 2018. URL: https://books.google.at/books?id=4EFKDwAAQBAJ.
  3. Christel Baier, Krishnendu Chatterjee, Tobias Meggendorfer, and Jakob Piribauer. Entropic risk for turn-based stochastic games. Information and Computation, 301:105214, 2024. URL: https://doi.org/10.1016/j.ic.2024.105214.
  4. Hans Wolfgang Brachinger. From variance to value at risk: A unified perspective on standardized risk measures. In Wolfgang Gaul and Hermann Locarek-Junge, editors, Classification in the Information Age, pages 91-99, Berlin, Heidelberg, 1999. Springer Berlin Heidelberg. Google Scholar
  5. Richard Bradley. Decision Theory: A Formal Philosophical Introduction, pages 611-655. Springer International Publishing, 2018. URL: https://doi.org/10.1007/978-3-319-77434-3_34.
  6. Véronique Bruyère, Emmanuel Filiot, Mickael Randour, and Jean-François Raskin. Meet your expectations with guarantees: Beyond worst-case synthesis in quantitative games. Information and Computation, 254:259-295, 2017. SR 2014. URL: https://doi.org/10.1016/j.ic.2016.10.011.
  7. Nicole Bäuerle and Ulrich Rieder. More risk-sensitive markov decision processes. Mathematics of Operations Research, 39(1):105-120, 2014. URL: https://doi.org/10.1287/MOOR.2013.0601.
  8. Luca de Alfaro, Thomas A. Henzinger, and Ranjit Jhala. Compositional methods for probabilistic systems. In CONCUR 2001 - Concurrency Theory, pages 351-365, Berlin, Heidelberg, 2001. Springer Berlin Heidelberg. URL: https://doi.org/10.1007/3-540-44685-0_24.
  9. Jerzy Filar and Lodewijk Kallenberg. Variance-penalized markov decision process. Mathematics of Operations Research, 14, February 1989. URL: https://doi.org/10.1287/moor.14.1.147.
  10. Hans Föllmer and Alexander Schied. Convex measures of risk and trading constraints. Finance and Stochastics, 6(4):429-447, October 2002. URL: https://doi.org/10.1007/s007800200072.
  11. Vojtěch Forejt, Marta Kwiatkowska, Gethin Norman, and David Parker. Automated Verification Techniques for Probabilistic Systems, pages 53-113. Springer Berlin Heidelberg, Berlin, Heidelberg, 2011. URL: https://doi.org/10.1007/978-3-642-21455-4_3.
  12. Hans Föllmer and Alexander Schied. Stochastic Finance: An Introduction in Discrete Time. Walter de Gruyter, Berlin, Boston, 4 edition, 2016. Google Scholar
  13. Jorge Gallego-Hernández and Alessio Mansutti. On the existential theory of the reals enriched with integer powers of a computable number. In Olaf Beyersdorff, Michal Pilipczuk, Elaine Pimentel, and Kim Thang Nguyen, editors, 42nd International Symposium on Theoretical Aspects of Computer Science, STACS 2025, March 4-7, 2025, Jena, Germany, volume 327 of LIPIcs, pages 37:1-37:18. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2025. URL: https://doi.org/10.4230/LIPICS.STACS.2025.37.
  14. Christoph Haase and Stefan Kiefer. The odds of staying on budget. In Automata, Languages, and Programming, pages 234-246, Berlin, Heidelberg, 2015. Springer Berlin Heidelberg. URL: https://doi.org/10.1007/978-3-662-47666-6_19.
  15. Ronald A. Howard and James E. Matheson. Risk-sensitive markov decision processes. Management Science, 18(7):356-369, 1972. Google Scholar
  16. Ronald A. Howard and James E. Matheson. Risk-sensitive markov decision processes. Management Science, 18(7):356-369, 1972. URL: http://www.jstor.org/stable/2629352.
  17. Leonid Hurwicz. Optimality criteria for decision-making under ignorance. Technical report, Cowles Commission for Research in Economics, 1951. URL: https://cowles.yale.edu/sites/default/files/2024-03/s-0355.pdf.
  18. Neil Immerman. Number of quantifiers is better than number of tape cells. J. Comput. Syst. Sci., 22:384-406, 1981. URL: https://doi.org/10.1016/0022-0000(81)90039-8.
  19. Jan Křetínský and Tobias Meggendorfer. Conditional value-at-risk for reachability and mean payoff in markov decision processes. In Proceedings of the 33rd Annual ACM/IEEE Symposium on Logic in Computer Science, LICS '18, pages 609-618. Association for Computing Machinery, 2018. URL: https://doi.org/10.1145/3209108.3209176.
  20. Claude Lefèvre. Optimal control of a birth and death epidemic process. Operations Research, 29(5):971-982, 1981. URL: https://doi.org/10.1287/OPRE.29.5.971.
  21. Shie Mannor and John N. Tsitsiklis. Mean-variance optimization in markov decision processes. In Proceedings of the 28th International Conference on Machine Learning, ICML 2011, pages 177-184. Omnipress, 2011. URL: https://icml.cc/2011/papers/156_icmlpaper.pdf.
  22. Tobias Meggendorfer. Risk-aware stochastic shortest path. Proceedings of the AAAI Conference on Artificial Intelligence, pages 9858-9867, 2022. URL: https://doi.org/10.1609/AAAI.V36I9.21222.
  23. John F. Nash. Equilibrium points in n-person games. Proceedings of the National Academy of Sciences, 36(1):48-49, 1950. URL: https://doi.org/10.1073/pnas.36.1.48.
  24. Andrzej S. Nowak. Notes on Risk-Sensitive Nash Equilibria, pages 95-109. Birkhäuser Boston, Boston, MA, 2005. URL: https://doi.org/10.1007/0-8176-4429-6_5.
  25. Martin J Osborne. An introduction to game theory. Oxford University Press google schola, 2:672-713, 2004. Google Scholar
  26. Bernardo K. Pagnoncelli, Oscar Dowson, and David P. Morton. Multistage stochastic programs with the entropic risk measure, 2020. URL: https://optimization-online.org/2020/08/7984/.
  27. Jakob Piribauer, Ocan Sankur, and Christel Baier. The variance-penalized stochastic shortest path problem. In 49th International Colloquium on Automata, Languages, and Programming, ICALP, volume 229 of LIPIcs, pages 129:1-129:19. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2022. URL: https://doi.org/10.4230/LIPICS.ICALP.2022.129.
  28. Mickael Randour, Jean-François Raskin, and Ocan Sankur. Variations on the stochastic shortest path problem. In Verification, Model Checking, and Abstract Interpretation, pages 1-18. Springer, 2015. URL: https://doi.org/10.1007/978-3-662-46081-8_1.
  29. Dawei Shi, Robert J Elliott, and Tongwen Chen. On finite-state stochastic modeling and secure estimation of cyber-physical systems. IEEE Transactions on Automatic Control, 62(1):65-80, 2016. URL: https://doi.org/10.1109/TAC.2016.2541919.
  30. Florent Teichteil-Königsbuch, Ugur Kuter, and Guillaume Infantes. Incremental plan aggregation for generating policies in MDPs. In Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: Volume 1 - Volume 1, AAMAS '10, pages 1231-1238. International Foundation for Autonomous Agents and Multiagent Systems, 2010. URL: https://dl.acm.org/citation.cfm?id=1838366.
  31. Michael Ummels and Dominik Wojtczak. The Complexity of Nash Equilibria in Stochastic Multiplayer Games. Logical Methods in Computer Science, Volume 7, Issue 3, September 2011. URL: https://doi.org/10.2168/LMCS-7(3:20)2011.
  32. Abraham Wald. Statistical Decision Functions. John Wiley & Sons, New York, 1950. Classic formulation of the minimax (Wald) criterion. Google Scholar
Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact