Time aggregated Markov decision processes via standard dynamic programming
Time aggregated Markov decision processes via standard dynamic programming
This note addresses the time aggregation approach to ergodic finite state Markov decision processes with uncontrollable states. We propose the use of the time aggregation approach as an intermediate step toward constructing a transformed MDP whose state space is comprised solely of the controllable states. The proposed approach simplifies the iterative search for the optimal solution by eliminating the need to define an equivalent parametric function, and results in a problem that can be solved by simpler, standard MDP algorithms.
Dynamic programming, Markov decision processes, Time aggregation
193-197
Arruda, Edilson F.
8eb3bd83-e883-4bf3-bfbc-7887c5daa911
Fragoso, Marcelo D.
7f484139-de97-4458-aa6b-dc3249811a08
1 May 2011
Arruda, Edilson F.
8eb3bd83-e883-4bf3-bfbc-7887c5daa911
Fragoso, Marcelo D.
7f484139-de97-4458-aa6b-dc3249811a08
Arruda, Edilson F. and Fragoso, Marcelo D.
(2011)
Time aggregated Markov decision processes via standard dynamic programming.
Operations Research Letters, 39 (3), .
(doi:10.1016/j.orl.2011.03.006).
Abstract
This note addresses the time aggregation approach to ergodic finite state Markov decision processes with uncontrollable states. We propose the use of the time aggregation approach as an intermediate step toward constructing a transformed MDP whose state space is comprised solely of the controllable states. The proposed approach simplifies the iterative search for the optimal solution by eliminating the need to define an equivalent parametric function, and results in a problem that can be solved by simpler, standard MDP algorithms.
This record has no associated files available for download.
More information
Published date: 1 May 2011
Keywords:
Dynamic programming, Markov decision processes, Time aggregation
Identifiers
Local EPrints ID: 445891
URI: http://eprints.soton.ac.uk/id/eprint/445891
ISSN: 0167-6377
PURE UUID: b9dd7d06-2bd0-4884-b9f3-3288f7ba8954
Catalogue record
Date deposited: 13 Jan 2021 17:31
Last modified: 18 Mar 2024 03:59
Export record
Altmetrics
Contributors
Author:
Edilson F. Arruda
Author:
Marcelo D. Fragoso
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics