Deep reinforcement learning with coalition action selection for online combinatorial resource allocation with arbitrary action space

Abstract

Current DRL algorithms typically assume a fixed number of possible
actions and sequentially select one action at a time, making them
inefficient for resource allocation problems with arbitrarily large action
spaces. Sequential action selection requires updating the state
for every action selected, which increases the depth of the decision,
the state space, the uncertainty, and the number of executions. This
affects the convergence of the algorithm and slows the execution
speed. Additionally, current DRL algorithms are not efficient for
online resource allocation problems with an arbitrary number of
task arrivals per time step because they assume a fixed number
of actions. To address these challenges, we propose a novel coalition
action selection approach that enables the DRL algorithm to
simultaneously select a coalition of an arbitrary number of actions
from a set with an arbitrary number of possible actions. By making
simultaneous decisions at each time step, coalition action selection
avoids the computational cost and large state space caused by the
sequential decision that updates the state multiple times. We evaluate
the performance and complexity of coalition action selection
and sequential action selection approaches using an online combinatorial
resource allocation problem. The results demonstrate that
the coalition action selection approach retains close performance
to the offline optimal for various online traffic demand arrival rates
of the online combinatorial resource allocation problem, while the
performance of the sequential action selection approach decreases
as the size of the problem increases. The experiments also demonstrate
that coalition action selection has much lower computational
complexity than sequential action selection.

More information

Identifiers

ORCID for Tesfay Zemuy Gebrekidan: orcid.org/0000-0002-0182-0997

ORCID for Sebastian Stein: orcid.org/0000-0003-2858-8857

ORCID for Timothy J. Norman: orcid.org/0000-0002-6387-4034

Catalogue record

Export record