Document Open Access Logo

A Generic Strategy Improvement Method for Simple Stochastic Games

Authors David Auger, Xavier Badin de Montjoye, Yann Strozecki

PDF

File

Thumbnail PDF
PDF
LIPIcs.MFCS.2021.12.pdf
  • Filesize: 0.76 MB
  • 22 pages

Document Identifiers

Related Versions

Subject Classification

ACM Subject Classification
  • Theory of computation → Algorithmic game theory
Keywords
  • Simple Stochastic Games
  • Strategy Improvement
  • Parametrized Complexity
  • Stopping
  • Meta Algorithm
  • f-strategy

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Abstract

We present a generic strategy improvement algorithm (GSIA) to find an optimal strategy of simple stochastic games (SSG). We prove the correctness of GSIA, and derive a general complexity bound, which implies and improves on the results of several articles. First, we remove the assumption that the SSG is stopping, which is usually obtained by a polynomial blowup of the game. Second, we prove a tight bound on the denominator of the values associated to a strategy, and use it to prove that all strategy improvement algorithms are in fact fixed parameter tractable in the number r of random vertices. All known strategy improvement algorithms can be seen as instances of GSIA, which allows to analyze the complexity of converge from below by Condon [Condon, 1993] and to propose a class of algorithms generalising Gimbert and Horn’s algorithm [Gimbert and Horn, 2008; Gimbert and Horn, 2009]. These algorithms terminate in at most r! iterations, and for binary SSGs, they do less iterations than the current best deterministic algorithm given by Ibsen-Jensen and Miltersen [Ibsen-Jensen and Miltersen, 2012].

Cite As Get BibTex

David Auger, Xavier Badin de Montjoye, and Yann Strozecki. A Generic Strategy Improvement Method for Simple Stochastic Games. In 46th International Symposium on Mathematical Foundations of Computer Science (MFCS 2021). Leibniz International Proceedings in Informatics (LIPIcs), Volume 202, pp. 12:1-12:22, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2021) https://doi.org/10.4230/LIPIcs.MFCS.2021.12

Author Details

David Auger
  • Université Paris Saclay, UVSQ, DAVID, France
Xavier Badin de Montjoye
  • Université Paris Saclay, UVSQ, DAVID, France
Yann Strozecki
  • Université Paris Saclay, UVSQ, DAVID, France

Acknowledgements

The authors want to thank Pierre Coucheney for many interesting discussions on SSGs.

References

  1. Daniel Andersson, Kristoffer Arnsfelt Hansen, Peter Bro Miltersen, and Troels Bjerre Sørensen. Deterministic graphical games revisited. In Conference on Computability in Europe, pages 1-10. Springer, 2008. Google Scholar
  2. Daniel Andersson and Peter Bro Miltersen. The complexity of solving stochastic games on graphs. In International Symposium on Algorithms and Computation, pages 112-121, 2009. Google Scholar
  3. David Auger, Xavier Badin de Montjoye, and Yann Strozecki. A generic strategy iteration method for simple stochastic games. arXiv preprint, 2021. URL: http://arxiv.org/abs/2102.04922.
  4. David Auger, Pierre Coucheney, and Yann Strozecki. Finding optimal strategies of almost acyclic simple stochastic games. In International Conference on Theory and Applications of Models of Computation, pages 67-85, 2014. Google Scholar
  5. David Auger, Pierre Coucheney, and Yann Strozecki. Solving Simple Stochastic Games with Few Random Nodes Faster Using Bland’s Rule. In 36th International Symposium on Theoretical Aspects of Computer Science (STACS 2019), pages 9:1-9:16, 2019. Google Scholar
  6. Cristian S Calude, Sanjay Jain, Bakhadyr Khoussainov, Wei Li, and Frank Stephan. Deciding parity games in quasi-polynomial time. SIAM Journal on Computing, 0(0):STOC17-152, 2020. Google Scholar
  7. Seth Chaiken and Daniel J Kleitman. Matrix tree theorems. Journal of combinatorial theory, Series A, 24(3):377-381, 1978. Google Scholar
  8. Krishnendu Chatterjee, Luca de Alfaro, and Thomas A. Henzinger. Strategy improvement for concurrent reachability and turn-based stochastic safety games. Journal of Computer and System Sciences, 79(5):640-657, 2013. Google Scholar
  9. Krishnendu Chatterjee and Nathanaël Fijalkow. A reduction from parity games to simple stochastic games. Electronic Proceedings in Theoretical Computer Science, 54, 2011. Google Scholar
  10. Krishnendu Chatterjee and Thomas A. Henzinger. Value Iteration, pages 107-138. Springer Berlin Heidelberg, 2008. Google Scholar
  11. Taolue Chen, Marta Kwiatkowska, Aistis Simaitis, and Clemens Wiltsche. Synthesis for multi-objective stochastic games: An application to autonomous urban driving. In Quantitative Evaluation of Systems, pages 322-337, 2013. Google Scholar
  12. Thomas Colcombet and Nathanaël Fijalkow. Universal graphs and good for games automata: New tools for infinite duration games. In International Conference on Foundations of Software Science and Computation Structures, pages 1-26. Springer, 2019. Google Scholar
  13. Anne Condon. The complexity of stochastic games. Information and Computation, 96(2):203-224, 1992. Google Scholar
  14. Anne Condon. On algorithms for simple stochastic games. Advances in computational complexity theory, 13:51-73, 1993. Google Scholar
  15. Decheng Dai and Rong Ge. New results on simple stochastic games. In International Symposium on Algorithms and Computation, pages 1014-1023. Springer, 2009. Google Scholar
  16. Hugo Gimbert and Florian Horn. Simple stochastic games with few random vertices are easy to solve. In Foundations of Software Science and Computational Structures, pages 5-19. Springer, 2008. Google Scholar
  17. Hugo Gimbert and Florian Horn. Solving simple stochastic games with few random vertices. Logical Methods in Computer Science, Volume 5, Issue 2, 2009. Google Scholar
  18. Rasmus Ibsen-Jensen and Peter Bro Miltersen. Solving simple stochastic games with few coin toss positions. In European Symposium on Algorithms, pages 636-647. Springer, 2012. Google Scholar
  19. Shunhua Jiang, Zhao Song, Omri Weinstein, and Hengjie Zhang. Faster dynamic matrix inverse for faster lps. arXiv preprint, 2020. URL: http://arxiv.org/abs/2004.07470.
  20. Brendan Juba. On the hardness of simple stochastic games. Master’s thesis, CMU, 2005. Google Scholar
  21. Jan Křetínsky, Emanuel Ramneantu, Alexander Slivinskiy, and Maximilian Weininger. Comparison of algorithms for simple stochastic games. arXiv preprint, 2020. URL: http://arxiv.org/abs/2009.10882.
  22. Walter Ludwig. A subexponential randomized algorithm for the simple stochastic game problem. Information and computation, 117(1):151-155, 1995. Google Scholar
  23. L. S. Shapley. Stochastic games. Proceedings of the National Academy of Sciences, 39(10):1095-1100, 1953. Google Scholar
  24. C Stirling. Bisimulation, modal logic and model checking games. Logic Journal of the IGPL, 7(1):103-124, 1999. Google Scholar
  25. Rahul Tripathi, Elena Valkanova, and VS Anil Kumar. On strategy improvement algorithms for simple stochastic games. Journal of Discrete Algorithms, 9(3):263-278, 2011. Google Scholar
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact