Opponent Indifference in Rating Systems: A Theoretical Case for Sonas

Authors Greg Bodwin, Forest Zhang



PDF
Thumbnail PDF

File

LIPIcs.ITCS.2023.21.pdf
  • Filesize: 0.78 MB
  • 21 pages

Document Identifiers

Author Details

Greg Bodwin
  • University of Michigan, Ann Arbor, MI, United States
Forest Zhang
  • University of Michigan, Ann Arbor, MI, United States

Acknowledgements

We are grateful to Jeff Sonas for a helpful discussion on the history and relationship between rating systems, and we are grateful to an anonymous reviewer for thorough and helpful feedback.

Cite AsGet BibTex

Greg Bodwin and Forest Zhang. Opponent Indifference in Rating Systems: A Theoretical Case for Sonas. In 14th Innovations in Theoretical Computer Science Conference (ITCS 2023). Leibniz International Proceedings in Informatics (LIPIcs), Volume 251, pp. 21:1-21:21, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2023)
https://doi.org/10.4230/LIPIcs.ITCS.2023.21

Abstract

In competitive games, it is common to assign each player a real number rating signifying their skill level. A rating system is a procedure by which player ratings are adjusted upwards each time they win, or downwards each time they lose. Many matchmaking systems give players some control over their opponent’s rating; for example, a player might be able to selectively initiate games against opponents whose ratings are publicly visible, or abort a game without penalty before it begins but after glimpsing their opponent’s rating. It is natural to ask whether one can design a rating system that does not incentivize a rating-maximizing player to act strategically, seeking games against opponents of one rating over another. We show the following: - The full version of this "opponent indifference" property is unfortunately too strong to be feasible. Although it is satisfied by some rating systems, these systems lack certain desirable expressiveness properties, suggesting that they are not suitable to capture most games of interest. - However, there is a natural relaxation, roughly requiring indifference between any two opponents who are both "reasonably evenly matched" with the choosing player. We prove that this relaxed variant of opponent indifference, which we call P opponent indifference, is viable. In fact, a certain strong version of P opponent indifference precisely characterizes the rating system Sonas, which was originally proposed for its empirical predictive accuracy on the outcomes of high-level chess games.

Subject Classification

ACM Subject Classification
  • Theory of computation → Algorithmic game theory and mechanism design
Keywords
  • Rating systems
  • opponent indifference
  • incentive compatibility
  • mechanism design
  • game theory

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads

References

  1. Universal rating system™. URL: http://universalrating.com/.
  2. Sharad Agarwal and Jacob R Lorch. Matchmaking for online games and other latency-sensitive p2p systems. In Proceedings of the ACM SIGCOMM 2009 conference on Data communication, pages 315-326, 2009. Google Scholar
  3. Josh Alman and Dylan McKay. Theoretical foundations of team matchmaking. In Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems, pages 1073-1081, 2017. Google Scholar
  4. Todd Bardwick. Csca informant – observations about chess rating distribution and progression. URL: https://coloradomasterchess.com/informant-ratings-and-expectations/.
  5. Shuo Chen and Thorsten Joachims. Modeling intransitivity in matchup and comparison data. In Proceedings of the ninth acm international conference on web search and data mining, pages 227-236, 2016. Google Scholar
  6. Zhengxing Chen, Yizhou Sun, Magy Seif El-Nasr, and Truong-Huy D Nguyen. Player skill decomposition in multiplayer online battle arenas. arXiv preprint, 2017. URL: http://arxiv.org/abs/1702.06253.
  7. Pierre Dangauthier, Ralf Herbrich, Tom Minka, and Thore Graepel. Trueskill through time: Revisiting the history of chess. Advances in neural information processing systems, 20, 2007. Google Scholar
  8. Aram Ebtekar and Paul Liu. Elo-mmr: A rating system for massive multiplayer competitions. In Proceedings of the Web Conference 2021, pages 1772-1784, 2021. Google Scholar
  9. Arpad E. Elo. The proposed uscf rating system, its development, theory, and applications. Chess Life, XXII(8):242-247, August 1967. Google Scholar
  10. Vitalii Emelianov, Nicolas Gast, and Patrick Loiseau. Fairness in selection problems with strategic candidates. arXiv preprint, 2022. URL: http://arxiv.org/abs/2205.12204.
  11. RNDr Michal Forišek. Theoretical and practical aspects of programming contest ratings.(2009), 2009. Google Scholar
  12. Mark E Glickman. A comprehensive guide to chess ratings. American Chess Journal, 3(1):59-102, 1995. Google Scholar
  13. Kenneth Harkness. The Official Chess Handbook. McKay, 1967. Google Scholar
  14. Ralf Herbrich, Tom Minka, and Thore Graepel. Trueskill™: a bayesian skill rating system. Advances in neural information processing systems, 19, 2006. Google Scholar
  15. Stephanie Kovalchik. Extension of the elo rating system to margin of victory. International Journal of Forecasting, 36(4):1329-1341, 2020. Google Scholar
  16. Lydia T Liu, Nikhil Garg, and Christian Borgs. Strategic ranking. In International Conference on Artificial Intelligence and Statistics, pages 2489-2518. PMLR, 2022. Google Scholar
  17. Joshua E Menke, C Shane Reese, and Tony R Martinez. Hierarchical models for estimating individual ratings from group competitions. In American Statistical Association. Citeseer, 2007. Google Scholar
  18. Tom Minka, Ryan Cleven, and Yordan Zaykov. Trueskill 2: An improved bayesian skill rating system. Technical Report, 2018. Google Scholar
  19. SERGEY I Nikolenko and ALEXANDER V Sirotkin. Extensions of the trueskilltm rating system. In proceedings of the 9th international conference on applications of fuzzy systems and soft computing, pages 151-160. Citeseer, 2010. Google Scholar
  20. Jeff Sonas. The sonas rating formula - better than elo? ChessBase, 2002. URL: https://en.chessbase.com/post/the-sonas-rating-formula-better-than-elo.
  21. Jeff Sonas. The elo rating system – correcting the expectancy tables. ChessBase, 2011. URL: https://en.chessbase.com/post/the-elo-rating-system-correcting-the-expectancy-tables.
  22. Jeff Sonas. What’s wrong with the elo system? ChessBase, 2020. URL: https://en.chessbase.com/post/what-s-wrong-with-the-elo-system.
  23. Weijie Su. You are the best reviewer of your own papers: An owner-assisted scoring mechanism. Advances in Neural Information Processing Systems, 34:27929-27939, 2021. Google Scholar
  24. Weijie J Su. A truthful owner-assisted scoring mechanism. arXiv preprint, 2022. URL: http://arxiv.org/abs/2206.08149.
  25. u/vlfph. Farming volatility: How a major flaw in a well-known rating system takes over the gbl leaderboard. URL: https://www.reddit.com/r/TheSilphRoad/comments/hwff2d/farming_volatility_how_a_major_flaw_in_a/.
  26. Lin Yang, Stanko Dimitrov, and Benny Mantin. Forecasting sales of new virtual goods with the elo rating system. Journal of Revenue and Pricing Management, 13(6):457-469, 2014. Google Scholar
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail