Optimal approximation schedules for iterative algorithms with application to dynamic programming

Almudevar, Anthony and Arruda, Edilson F. (2007) Optimal approximation schedules for iterative algorithms with application to dynamic programming. In Proceedings of the 46th IEEE Conference on Decision and Control 2007, CDC. pp. 4087-4094 . (doi:10.1109/CDC.2007.4434681).

Record type: Conference or Workshop Item (Paper)

Abstract

Many iterative algorithms rely on operators which may be difficult or impossible to evaluate exactly, but for which approximations are available. Furthermore, a graduated range of approximations may be available, inducing a functional relationship between computational complexity and approximation tolerance. In such a case, a reasonable strategy would be to vary tolerance over iterations, starting with a cruder approximation, then gradually decreasing tolerance as the solution is approached. In this article, it is shown that under general conditions, for linearly convergent algorithms the optimal choice of approximation tolerance convergence rate is the same linear convergence rate as the exact algorithm itself, regardless of the tolerance/complexity relationship. We illustrate this result by presenting a partial information value iteration (PIVI) algorithm for discrete time dynamic programming problems. The proposed algorithm makes use of increasingly accurate partial model information in order to decrease the computational burden of the standard value iteration algorithm. The algorithm is applied to a stochastic network example and compared to value iteration for the purpose of benchmarking.

This record has no associated files available for download.