,
Mickael Randour
Creative Commons Attribution 4.0 International license
We consider multi-dimensional payoff functions in partially observable Markov decision processes. We study the structure of the set of expected payoff vectors of all strategies (policies) and study what kind are needed to achieve a given expected payoff vector. In general, pure strategies (i.e., not resorting to randomisation) do not suffice for this problem. We prove that for any payoff function for which the expectation is well-defined under all strategies, it is sufficient to mix (i.e., randomly select a pure strategy at the start of a play and committing to it for the rest of the play) finitely many pure strategies to approximate any expected payoff vector up to any precision. Furthermore, for any payoff function for which the expected payoff is finite under all strategies, any expected payoff vector can be obtained exactly by mixing finitely many strategies. These results are already novel in the context of (perfect information) Markov decision processes.
@InProceedings{main_et_al:LIPIcs.LICS.2026.68,
author = {Main, James C. A. and Randour, Mickael},
title = {{Mixing Any Cocktail with Limited Ingredients: On the Structure of Payoff Sets in Multi-Objective POMDPs and Its Impact on Randomised Strategies}},
booktitle = {41st Annual Symposium on Logic in Computer Science (LICS 2026)},
pages = {68:1--68:27},
series = {Leibniz International Proceedings in Informatics (LIPIcs)},
ISBN = {978-3-95977-434-5},
ISSN = {1868-8969},
year = {2026},
volume = {380},
editor = {Faggian, Claudia and Katoen, Joost-Pieter},
publisher = {Schloss Dagstuhl -- Leibniz-Zentrum f{\"u}r Informatik},
address = {Dagstuhl, Germany},
URL = {https://drops.dagstuhl.de/entities/document/10.4230/LIPIcs.LICS.2026.68},
URN = {urn:nbn:de:0030-drops-268559},
doi = {10.4230/LIPIcs.LICS.2026.68},
annote = {Keywords: partially observable Markov decision process, multiple objectives, randomised strategies, mixed strategies, strategy complexity}
}