The University of Southampton
University of Southampton Institutional Repository

Optimal approximation schedules for a class of iterative algorithms, with an application to multigrid value iteration

Optimal approximation schedules for a class of iterative algorithms, with an application to multigrid value iteration
Optimal approximation schedules for a class of iterative algorithms, with an application to multigrid value iteration

Many iterative algorithms employ operators which are difficult to evaluate exactly, but for which a graduated range of approximations exist. In such cases, coarse-to-fine algorithms are often used, in which a crude initial operator approximation is gradually refined with new iterations. In such algorithms, because the computational complexity increases over iterations, the algorithm's convergence rate is properly calculated with respect to cumulative computation time. This suggests the problem of determining an optimal rate of refinement for the operator approximation. This paper shows that, for linearly convergent algorithm, the optimal rate of refinement approaches the rate of convergence of the exact algorithm itself, regardless of the tolerance-complexity relationship. We illustrate this result with an analysis of coarse-to-fine grid algorithms for Markov decision processes with continuous state spaces. Using the methods proposed here we deduce an algorithm that presents optimal complexity results and consists solely of a non-adaptive schedule for the gradual decrease of grid size.

Approximate value iteration, Markov and semi-Markov decision processes, numerical approximation
0018-9286
3132-3146
Almudevar, Anthony
f0998a97-a377-41a9-82d0-0c1de5f33688
De Arruda, Edilson Fernandes
8eb3bd83-e883-4bf3-bfbc-7887c5daa911
Almudevar, Anthony
f0998a97-a377-41a9-82d0-0c1de5f33688
De Arruda, Edilson Fernandes
8eb3bd83-e883-4bf3-bfbc-7887c5daa911

Almudevar, Anthony and De Arruda, Edilson Fernandes (2012) Optimal approximation schedules for a class of iterative algorithms, with an application to multigrid value iteration. IEEE Transactions on Automatic Control, 57 (12), 3132-3146, [6213075]. (doi:10.1109/TAC.2012.2203053).

Record type: Article

Abstract

Many iterative algorithms employ operators which are difficult to evaluate exactly, but for which a graduated range of approximations exist. In such cases, coarse-to-fine algorithms are often used, in which a crude initial operator approximation is gradually refined with new iterations. In such algorithms, because the computational complexity increases over iterations, the algorithm's convergence rate is properly calculated with respect to cumulative computation time. This suggests the problem of determining an optimal rate of refinement for the operator approximation. This paper shows that, for linearly convergent algorithm, the optimal rate of refinement approaches the rate of convergence of the exact algorithm itself, regardless of the tolerance-complexity relationship. We illustrate this result with an analysis of coarse-to-fine grid algorithms for Markov decision processes with continuous state spaces. Using the methods proposed here we deduce an algorithm that presents optimal complexity results and consists solely of a non-adaptive schedule for the gradual decrease of grid size.

This record has no associated files available for download.

More information

Published date: 7 December 2012
Keywords: Approximate value iteration, Markov and semi-Markov decision processes, numerical approximation

Identifiers

Local EPrints ID: 445907
URI: http://eprints.soton.ac.uk/id/eprint/445907
ISSN: 0018-9286
PURE UUID: 660fb1e5-275a-40ec-8255-cedfd867f1c7
ORCID for Edilson Fernandes De Arruda: ORCID iD orcid.org/0000-0002-9835-352X

Catalogue record

Date deposited: 13 Jan 2021 17:31
Last modified: 18 Mar 2024 03:59

Export record

Altmetrics

Contributors

Author: Anthony Almudevar
Author: Edilson Fernandes De Arruda ORCID iD

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×