Standard dynamic programming applied to time aggregated Markov decision processes

Arruda, Edilson F. and Fragoso, Marcelo D. (2009) Standard dynamic programming applied to time aggregated Markov decision processes. In Proceedings of the 48th IEEE Conference on Decision and Control held jointly with 2009 28th Chinese Control Conference, CDC/CCC 2009. pp. 2576-2580 . (doi:10.1109/CDC.2009.5400692).

Record type: Conference or Workshop Item (Paper)

Abstract

In this note we address the time aggregation approach to ergodic finite state Markov decision processes with uncontrollable states. We propose the use of the time aggregation approach as an intermediate step toward constructing a transformed MDP whose state space is comprised solely of the controllable states. The proposed approach simplifies the iterative search for the optimal solution by eliminating the need to define an equivalent parametric function, and results in a problem that can be solved by simpler, standard MDP algorithms.

This record has no associated files available for download.

More information

Published date: 1 December 2009

Venue - Dates: 48th IEEE Conference on Decision and Control held jointly with 2009 28th Chinese Control Conference, CDC/CCC 2009, , Shanghai, China, 2009-12-15 - 2009-12-18

Keywords: Dynamic programing, Markov decision processes, Time aggregation