Boker, Udi ;
Hefetz, Guy
DiscountedSum Automata with Multiple Discount Factors
Abstract
Discounting the influence of future events is a key paradigm in economics and it is widely used in computerscience models, such as games, Markov decision processes (MDPs), reinforcement learning, and automata. While a single game or MDP may allow for several different discount factors, discountedsum automata (NDAs) were only studied with respect to a single discount factor. For every integer λ ∈ ℕ⧵{0,1}, as opposed to every λ ∈ ℚ⧵ℕ, the class of NDAs with discount factor λ (λNDAs) has good computational properties: it is closed under determinization and under the algebraic operations min, max, addition, and subtraction, and there are algorithms for its basic decision problems, such as automata equivalence and containment.
We define and analyze discountedsum automata in which each transition can have a different integral discount factor (integral NMDAs). We show that integral NMDAs with an arbitrary choice of discount factors are not closed under determinization and under algebraic operations. We then define and analyze a restricted class of integral NMDAs, which we call tidy NMDAs, in which the choice of discount factors depends on the prefix of the word read so far. Tidy NMDAs are as expressive as deterministic integral NMDAs with an arbitrary choice of discount factors, and some of their special cases are NMDAs in which the discount factor depends on the action (alphabet letter) or on the elapsed time.
We show that for every function θ that defines the choice of discount factors, the class of θNMDAs enjoys all of the above good properties of integral NDAs, as well as the same complexities of the required decision problems. To this end, we also improve the previously known complexities of the decision problems of integral NDAs, and present tight bounds on the size blowup involved in algebraic operations on them.
All our results hold equally for automata on finite words and for automata on infinite words.
BibTeX  Entry
@InProceedings{boker_et_al:LIPIcs:2021:13446,
author = {Udi Boker and Guy Hefetz},
title = {{DiscountedSum Automata with Multiple Discount Factors}},
booktitle = {29th EACSL Annual Conference on Computer Science Logic (CSL 2021)},
pages = {12:112:23},
series = {Leibniz International Proceedings in Informatics (LIPIcs)},
ISBN = {9783959771757},
ISSN = {18688969},
year = {2021},
volume = {183},
editor = {Christel Baier and Jean GoubaultLarrecq},
publisher = {Schloss DagstuhlLeibnizZentrum f{\"u}r Informatik},
address = {Dagstuhl, Germany},
URL = {https://drops.dagstuhl.de/opus/volltexte/2021/13446},
URN = {urn:nbn:de:0030drops134468},
doi = {10.4230/LIPIcs.CSL.2021.12},
annote = {Keywords: Automata, Discountedsum, Quantitative verification, NMDA, NDA}
}
13.01.2021
Keywords: 

Automata, Discountedsum, Quantitative verification, NMDA, NDA 
Seminar: 

29th EACSL Annual Conference on Computer Science Logic (CSL 2021)

Issue date: 

2021 
Date of publication: 

13.01.2021 