Deep reinforcement learning with coalition action selection for online combinatorial resource allocation with arbitrary action space

Abstract

Current DRL algorithms typically assume a fixed number of possible actions and sequentially select one action at a time, which makes them inefficient for resource allocation problems with arbitrarily large action spaces. Sequential action selection requires updating the state for every action selected, which increases the depth of the decision, the state space, the uncertainty, and the number of executions. This affects the convergence of the algorithm and slows the execution speed. Additionally, current DRL algorithms are not efficient for online resource allocation problems with an arbitrary number of task arrivals per time step because they assume a fixed number of actions. To address these challenges, we propose a novel coalition action selection approach that enables the DRL algorithm to simultaneously select a coalition of an arbitrary number of actions, from a set with an arbitrary number of possible actions. By making simultaneous decisions at each time step, coalition action selection avoids the need to update the state multiple times. We evaluate the performance and complexity of coalition action selection and sequential action selection approaches using an online combinatorial resource allocation problem. The results demonstrate that the coalition action selection approach retains close performance to the offline optimal for various online traffic demand arrival rates of the online combinatorial resource allocation problem, while the performance of the sequential action selection approach decreases as the size of the problem increases. The experiments also demonstrate that coalition action selection has much lower computational complexity than sequential action selection.

More information

Identifiers

ORCID for Sebastian Stein: orcid.org/0000-0003-2858-8857

ORCID for Timothy J. Norman: orcid.org/0000-0002-6387-4034

Catalogue record

Export record