Document Open Access Logo

Incentivizing High-Quality Content in Online Recommender Systems

Authors Xinyan Hu , Meena Jagadeesan , Michael I. Jordan , Jacob Steinhardt

PDF

File

Thumbnail PDF
PDF
LIPIcs.FORC.2026.15.pdf
  • Filesize: 0.87 MB
  • 18 pages

Document Identifiers

Related Versions

Subject Classification

ACM Subject Classification
  • Theory of computation → Algorithmic game theory
  • Information systems → Recommender systems
  • Theory of computation → Online learning algorithms
Keywords
  • recommender systems
  • content quality
  • producer incentives
  • online learning
  • algorithmic game theory
  • Stackelberg games

Metrics

Abstract

In content recommender systems such as TikTok and YouTube, the platform’s recommendation algorithm shapes content producer incentives. Many platforms employ online learning, which generates intertemporal incentives, since content produced today affects recommendations of future content. We study the game between producers and analyze the content created at equilibrium. We prove that standard online learning algorithms, such as Hedge and EXP3, unfortunately incentivize producers to create low-quality content, where producers' effort approaches zero in the long run for typical learning rate schedules. Motivated by this negative result, we design learning algorithms that incentivize producers to invest high effort and achieve high user welfare. At a conceptual level, our work illustrates the unintended impact that a platform’s learning algorithm can have on content quality and introduces algorithmic approaches to mitigating these effects.

Cite As Get BibTex

Xinyan Hu, Meena Jagadeesan, Michael I. Jordan, and Jacob Steinhardt. Incentivizing High-Quality Content in Online Recommender Systems. In 7th Symposium on Foundations of Responsible Computing (FORC 2026). Leibniz International Proceedings in Informatics (LIPIcs), Volume 368, pp. 15:1-15:18, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2026) https://doi.org/10.4230/LIPIcs.FORC.2026.15

Author Details

Xinyan Hu
  • University of California, Berkeley, CA, USA
Meena Jagadeesan
  • Stanford University, CA, USA
Michael I. Jordan
  • University of California, Berkeley, CA, USA
  • Inria and École Normale Supérieure, Paris, France
Jacob Steinhardt
  • University of California, Berkeley, CA, USA
  • Transluce, San Francisco, CA, USA

Acknowledgements

Xinyan Hu and Meena Jagadeesan contributed equally to this work.

References

  1. Yasin Abbasi-Yadkori, Dávid Pál, and Csaba Szepesvári. Improved algorithms for linear stochastic bandits. In Advances in Neural Information Processing Systems 24: 25th Annual Conference on Neural Information Processing Systems 2011. Proceedings of a meeting held 12-14 December 2011, Granada, Spain, pages 2312-2320, 2011. URL: https://proceedings.neurips.cc/paper/2011/hash/e1d5be1c7f2f456670de3d53c7b54f4a-Abstract.html.
  2. Naoki Abe and Philip M. Long. Associative reinforcement learning using linear probabilistic concepts. In Proceedings of the Sixteenth International Conference on Machine Learning (ICML 1999), Bled, Slovenia, June 27 - 30, 1999, pages 3-11. Morgan Kaufmann, 1999. Google Scholar
  3. Maria-Florina Balcan, Avrim Blum, Nika Haghtalab, and Ariel D Procaccia. Commitment without regrets: Online learning in Stackelberg security games. In Proceedings of the 16th ACM Conference on Economics and Computation, pages 61-78, 2015. URL: https://doi.org/10.1145/2764468.2764478.
  4. Ran Ben Basat, Moshe Tennenholtz, and Oren Kurland. A game theoretic analysis of the adversarial retrieval setting. J. Artif. Int. Res., 60(1):1127-1164, 2017. URL: https://doi.org/10.1613/JAIR.5547.
  5. Omer Ben-Porat, Itay Rosenberg, and Moshe Tennenholtz. Content provider dynamics and coordination in recommendation ecosystems. In Advances in Neural Information Processing Systems (NeurIPS), 2020. Google Scholar
  6. Omer Ben-Porat and Moshe Tennenholtz. A game-theoretic approach to recommendation systems with strategic content providers. In Advances in Neural Information Processing Systems (NeurIPS), pages 1118-1128, 2018. URL: https://proceedings.neurips.cc/paper/2018/hash/a9a1d5317a33ae8cef33961c34144f84-Abstract.html.
  7. Omer Ben-Porat and Rotem Torkan. Learning with exposure constraints in recommendation systems. In Proceedings of the ACM Web Conference 2023, WWW 2023, Austin, TX, USA, 30 April 2023 - 4 May 2023, pages 3456-3466. ACM, 2023. URL: https://doi.org/10.1145/3543507.3583320.
  8. Micah D. Carroll, Anca D. Dragan, Stuart Russell, and Dylan Hadfield-Menell. Estimating and penalizing induced preference shifts in recommender systems. In International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pages 2686-2708. PMLR, 2022. URL: https://proceedings.mlr.press/v162/carroll22a.html.
  9. Yiling Chen, Yang Liu, and Chara Podimata. Learning strategy-aware linear classifiers. Advances in Neural Information Processing Systems, 33:15265-15276, 2020. Google Scholar
  10. Natalie Collina, Eshwar Ram Arunachaleswaran, and Michael Kearns. Efficient stackelberg strategies for finitely repeated games. In Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2023, London, United Kingdom, 29 May 2023 - 2 June 2023, pages 643-651. ACM, 2023. URL: https://doi.org/10.5555/3545946.3598695.
  11. Sarah Dean and Jamie Morgenstern. Preference dynamics under personalized recommendations. In EC '22: The 23rd ACM Conference on Economics and Computation, Boulder, CO, USA, July 11 - 15, 2022, pages 795-816. ACM, 2022. URL: https://doi.org/10.1145/3490486.3538346.
  12. Jinshuo Dong, Aaron Roth, Zachary Schutzman, Bo Waggoner, and Zhiwei Steven Wu. Strategic classification from revealed preferences. In Proceedings of the 2018 ACM Conference on Economics and Computation, pages 55-70, 2018. URL: https://doi.org/10.1145/3219166.3219193.
  13. Itay Eilat and Nir Rosenfeld. Performative recommendation: Diversifying content via strategic incentives. CoRR, abs/2302.04336, 2023. URL: https://doi.org/10.48550/arXiv.2302.04336.
  14. Arpita Ghosh and Patrick Hummel. Learning and incentives in user-generated content: multi-armed bandits with endogenous arms. In Innovations in Theoretical Computer Science, ITCS '13, Berkeley, CA, USA, January 9-12, 2013, pages 233-246. ACM, 2013. URL: https://doi.org/10.1145/2422436.2422465.
  15. Arpita Ghosh and R. Preston McAfee. Incentivizing high-quality user-generated content. In Proceedings of the 20th International Conference on World Wide Web, WWW 2011, Hyderabad, India, March 28 - April 1, 2011, pages 137-146. ACM, 2011. URL: https://doi.org/10.1145/1963405.1963428.
  16. Tarleton Gillespie. Custodians of the Internet: Platforms, Content Moderation, and the Hidden Decisions That Shape Social Media. Yale University Press, New Haven, 2018. Google Scholar
  17. Nika Haghtalab, Thodoris Lykouris, Sloan Nietert, and Alex Wei. Learning in stackelberg games with non-myopic agents. CoRR, abs/2208.09407, 2022. URL: https://doi.org/10.48550/arXiv.2208.09407.
  18. Moritz Hardt, Nimrod Megiddo, Christos Papadimitriou, and Mary Wootters. Strategic classification. In Proceedings of the 7th Conference on Innovations in Theoretical Computer Science (ITCS), pages 111-122, 2016. URL: https://doi.org/10.1145/2840728.2840730.
  19. Andreas A. Haupt, Dylan Hadfield-Menell, and Chara Podimata. Recommending to strategic users. CoRR, abs/2302.06559, 2023. URL: https://doi.org/10.48550/arXiv.2302.06559.
  20. Elad Hazan. Introduction to online convex optimization. CoRR, abs/1909.05207, 2019. URL: https://arxiv.org/abs/1909.05207.
  21. Jiri Hron, Karl Krauth, Michael I. Jordan, Niki Kilbertus, and Sarah Dean. Modeling content creator incentives on algorithm-curated platforms. CoRR, abs/2206.13102, 2022. URL: https://doi.org/10.48550/arXiv.2206.13102.
  22. Xinyan Hu, Meena Jagadeesan, Michael I. Jordan, and Jacob Steinhardt. Incentivizing high-quality content in online recommender systems. CoRR, abs/2306.07479, 2023. Full version with all proofs. URL: https://doi.org/10.48550/arXiv.2306.07479.
  23. Yifan Hu, Yehuda Koren, and Chris Volinsky. Collaborative filtering for implicit feedback datasets. In Proceedings of the 8th IEEE International Conference on Data Mining (ICDM 2008), December 15-19, 2008, Pisa, Italy, pages 263-272. IEEE Computer Society, 2008. URL: https://doi.org/10.1109/ICDM.2008.22.
  24. Meena Jagadeesan, Nikhil Garg, and Jacob Steinhardt. Supply-side equilibria in recommender systems. CoRR, abs/2206.13489, 2022. URL: https://doi.org/10.48550/arXiv.2206.13489.
  25. Debarun Kar, Thanh H. Nguyen, Fei Fang, Matthew Brown, Arunesh Sinha, Milind Tambe, and Albert Xin Jiang. Trends and Applications in Stackelberg Security Games, pages 1223-1269. Springer International Publishing, Cham, 2018. Google Scholar
  26. Ilan Kremer, Yishay Mansour, and Motty Perry. Implementing the "wisdom of the crowd". In Proceedings of the fourteenth ACM Conference on Electronic Commerce, EC 2013, pages 605-606, 2013. URL: https://doi.org/10.1145/2492002.2482542.
  27. Tao Li and Suresh P. Sethi. A review of dynamic Stackelberg game models. Discrete and Continuous Dynamical Systems - B, 22(1):125-159, 2017. Google Scholar
  28. Yang Liu and Chien-Ju Ho. Incentivizing high quality user contributions: New arm generation in bandit learning. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, pages 1146-1153. AAAI Press, 2018. URL: https://doi.org/10.1609/AAAI.V32I1.11464.
  29. Martin Mladenov, Elliot Creager, Omer Ben-Porat, Kevin Swersky, Richard Zemel, and Craig Boutilier. Optimizing long-term social welfare in recommender systems: A constrained matching approach, 2020. URL: https://arxiv.org/abs/2008.00104.
  30. Gergely Neu and Julia Olkhovskaya. Efficient and robust algorithms for adversarial linear contextual bandits. In Conference on Learning Theory, COLT 2020, 9-12 July 2020, Virtual Event [Graz, Austria], volume 125 of Proceedings of Machine Learning Research, pages 3049-3068. PMLR, 2020. URL: http://proceedings.mlr.press/v125/neu20b.html.
  31. Alexander Rakhlin and Karthik Sridharan. BISTRO: an efficient relaxation-based method for contextual bandits. In Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016, volume 48, pages 1977-1985, 2016. URL: http://proceedings.mlr.press/v48/rakhlin16.html.
  32. Goldman Sachs. The creator economy could approach half-a-trillion dollars by 2027, 2023. Google Scholar
  33. Vasilis Syrgkanis, Haipeng Luo, Akshay Krishnamurthy, and Robert E. Schapire. Improved regret bounds for oracle-based adversarial contextual bandits. In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, pages 3135-3143, 2016. URL: https://proceedings.neurips.cc/paper/2016/hash/dfa92d8f817e5b08fcaafb50d03763cf-Abstract.html.
  34. Jian Wei, Jianhua He, Kai Chen, Yi Zhou, and Zuoyin Tang. Collaborative filtering and deep learning based recommendation system for cold start items. Expert Syst. Appl., 69:29-39, 2017. URL: https://doi.org/10.1016/J.ESWA.2016.09.040.
  35. Ji Yang, Xinyang Yi, Derek Zhiyuan Cheng, Lichan Hong, Yang Li, Simon Xiaoming Wang, Taibai Xu, and Ed H. Chi. Mixed negative sampling for learning two-tower neural networks in recommendations. In Companion of The 2020 Web Conference 2020, Taipei, Taiwan, April 20-24, 2020, pages 441-447. ACM / IW3C2, 2020. URL: https://doi.org/10.1145/3366424.3386195.
  36. Fan Yao, Chuanhao Li, Denis Nekipelov, Hongning Wang, and Haifeng Xu. How bad is top-k recommendation under competing content creators? CoRR, abs/2302.01971, 2023. URL: https://doi.org/10.48550/arXiv.2302.01971.
  37. Xinyang Yi, Ji Yang, Lichan Hong, Derek Zhiyuan Cheng, Lukasz Heldt, Aditee Kumthekar, Zhe Zhao, Li Wei, and Ed H. Chi. Sampling-bias-corrected neural modeling for large corpus item recommendations. In Proceedings of the 13th ACM Conference on Recommender Systems, RecSys 2019, Copenhagen, Denmark, September 16-20, 2019, pages 269-277. ACM, 2019. URL: https://doi.org/10.1145/3298689.3346996.
  38. Mengmeng Yu and Seung Ho Hong. Supply–demand balancing for power management in smart grid: A stackelberg game approach. Applied Energy, 164:702-710, 2016. Google Scholar
  39. Banghua Zhu, Stephen Bates, Zhuoran Yang, Yixin Wang, Jiantao Jiao, and Michael I. Jordan. The sample complexity of online contract design. CoRR, abs/2211.05732, 2022. URL: https://doi.org/10.48550/arXiv.2211.05732.
  40. Tijana Zrnic, Eric Mazumdar, S. Shankar Sastry, and Michael I. Jordan. Who leads and who follows in strategic classification? In Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, pages 15257-15269, 2021. URL: https://proceedings.neurips.cc/paper/2021/hash/812214fb8e7066bfa6e32c626c2c688b-Abstract.html.
Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2026 Schloss Dagstuhl – LZI GmbH Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact