The University of Southampton
University of Southampton Institutional Repository

Multi-agent deep reinforcement learning based cooperative edge caching for ultra-dense next-generation networks

Multi-agent deep reinforcement learning based cooperative edge caching for ultra-dense next-generation networks
Multi-agent deep reinforcement learning based cooperative edge caching for ultra-dense next-generation networks
The soaring mobile data traffic demands have spawned the innovative concept of mobile edge caching in ultradense next-generation networks, which mitigates their heavy traffic burden. We conceive cooperative content sharing between base stations (BSs) for improving the exploitation of the limited storage of a single edge cache. We formulate the cooperative caching problem as a partially observable Markov decision
process (POMDP) based multi-agent decision problem, which jointly optimizes the costs of fetching contents from the local BS, from the nearby BSs and from the remote servers. To solve this problem, we devise a multi-agent actor-critic framework, where a communication module is introduced to extract and share the variability of the actions and observations of all BSs. To beneficially exploit the spatio-temporal differences of the content popularity, we harness a variational recurrent neural network (VRNN) for estimating the time-variant popularity distribution in each BS. Based on multi-agent deep reinforcement learning, we conceive a cooperative edge caching algorithm where the BSs operate cooperatively, since the distributed decision making of each agent depends on both the local and the global states. Our experiments conducted within a large scale cellular network having numerous BSs reveal that the proposed algorithm relying on the collaboration of BSs substantially improves the benefits of edge caches.
0090-6778
Chen, Shuangwu
7365b26b-794f-40f1-b620-df39462626aa
Yao, Zhen
bd976a1d-25fd-479b-995d-2875a456d42b
Jiang, Xiaofeng
75576bde-d011-413b-b1b9-7659dab71771
Yang, Jian
a95e75db-6340-4085-af35-54e655b46b6f
Hanzo, Lajos
66e7266f-3066-4fc0-8391-e000acce71a1
Chen, Shuangwu
7365b26b-794f-40f1-b620-df39462626aa
Yao, Zhen
bd976a1d-25fd-479b-995d-2875a456d42b
Jiang, Xiaofeng
75576bde-d011-413b-b1b9-7659dab71771
Yang, Jian
a95e75db-6340-4085-af35-54e655b46b6f
Hanzo, Lajos
66e7266f-3066-4fc0-8391-e000acce71a1

Chen, Shuangwu, Yao, Zhen, Jiang, Xiaofeng, Yang, Jian and Hanzo, Lajos (2020) Multi-agent deep reinforcement learning based cooperative edge caching for ultra-dense next-generation networks. IEEE Transactions on Communications. (In Press)

Record type: Article

Abstract

The soaring mobile data traffic demands have spawned the innovative concept of mobile edge caching in ultradense next-generation networks, which mitigates their heavy traffic burden. We conceive cooperative content sharing between base stations (BSs) for improving the exploitation of the limited storage of a single edge cache. We formulate the cooperative caching problem as a partially observable Markov decision
process (POMDP) based multi-agent decision problem, which jointly optimizes the costs of fetching contents from the local BS, from the nearby BSs and from the remote servers. To solve this problem, we devise a multi-agent actor-critic framework, where a communication module is introduced to extract and share the variability of the actions and observations of all BSs. To beneficially exploit the spatio-temporal differences of the content popularity, we harness a variational recurrent neural network (VRNN) for estimating the time-variant popularity distribution in each BS. Based on multi-agent deep reinforcement learning, we conceive a cooperative edge caching algorithm where the BSs operate cooperatively, since the distributed decision making of each agent depends on both the local and the global states. Our experiments conducted within a large scale cellular network having numerous BSs reveal that the proposed algorithm relying on the collaboration of BSs substantially improves the benefits of edge caches.

Text
final-manuscript - Accepted Manuscript
Restricted to Repository staff only until 4 February 2021.
Request a copy

More information

Accepted/In Press date: 5 December 2020

Identifiers

Local EPrints ID: 445630
URI: http://eprints.soton.ac.uk/id/eprint/445630
ISSN: 0090-6778
PURE UUID: d21f952b-0d40-4e47-beb0-f5389e8b71de
ORCID for Lajos Hanzo: ORCID iD orcid.org/0000-0002-2636-5214

Catalogue record

Date deposited: 04 Jan 2021 17:33
Last modified: 04 Jan 2021 17:33

Export record

Contributors

Author: Shuangwu Chen
Author: Zhen Yao
Author: Xiaofeng Jiang
Author: Jian Yang
Author: Lajos Hanzo ORCID iD

University divisions

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×