2026-08-16T23:04:18Z https://drops.dagstuhl.de/oai

oai:drops-oai.dagstuhl.de:1403 2024-03-06T11:07:46Z ddc:004 open_access

A policy iteration algorithm for Markov decision processes skip-free in one direction Lambert, Joke van Houdt, Benny Blondia, Chris Markov Decision Process Policy Evaluation Skip-Free Optical buffers Fibre Delay Lines In this paper we present a new algorithm for policy iteration for Markov decision processes (MDP) skip-free in one direction. This algorithm, which is based on matrix analytic methods, is in the same spirit as the algorithm of White (Stochastic Models, 21:785-797, 2005) which was limited to matrices that are skip-free in both directions. Optimization problems that can be solved using Markov decision processes arise in the domain of optical buffers, when trying to improve loss rates of fibre delay line (FDL) buffers. Based on the analysis of such an FDL buffer we present a comparative study between the different techniques available to solve an MDP. The results illustrate that the exploitation of the structure of the transition matrices places us in a position to deal with larger systems, while reducing the computation times. Schloss Dagstuhl – Leibniz-Zentrum für Informatik Joke Lambert and Benny van Houdt and Chris Blondia 2008 Is Part Of Dagstuhl Seminar Proceedings, Volume 7461, Numerical Methods for Structured Markov Chains (2008) InProceedings Text doc-type:ResearchArticle publishedVersion application/pdf doi:10.4230/DagSemProc.07461.3 urn:nbn:de:0030-drops-14032 https://drops.dagstuhl.de/entities/document/10.4230/DagSemProc.07461.3 eng https://creativecommons.org/licenses/by/4.0/legalcode