Adaptive market making using reinforcement learning in a multi-agent market simulation

Abstract

Financial markets are amongst the most complex and well-studied multi-agent systems. Over the past several decades, researchers have devoted considerable effort to understanding their structural and behavioural properties. However, building a simulated market environment that can replicate realistic interactive behaviour remains an ongoing challenge. This limitation restricts the scope and validity of experiments aimed at testing new trading strategies or hypotheses, particularly when simplifying assumptions, such as negligible market impact or the absence of market feedback, fail to hold.

This thesis addresses this challenge by building on the Agent-Based Interactive Discrete Event Simulation (ABIDES) simulation platform to create an empirically grounded and reproducible environment for studying cryptocurrency markets. First, we develop a methodology for tuning simulation parameters to replicate the stylised facts observed in Binance following the 2017 cryptocurrency boom. Second, we introduce Price-Reverting Impact Model of a cryptocurrency Exchange (PRIME), a novel configuration of the ABIDES simulator designed to produce realistic market responses to agent actions whilst retaining the ability to track an external price series. Third, we use the PRIME framework to conduct the first controlled comparison of Reinforcement Learning (RL) based market makers across various architectures in a realistic, interactive environment, establishing a benchmark for future research.

Our findings demonstrate that a carefully calibrated agent population comprising Zero Intelligence, Momentum, and Mean-Reversion agents can reproduce key statistical properties observed in the Bitcoin market. Furthermore, the PRIME framework successfully captures market impact behaviours described in existing and novel empirical observations, allowing for controlled but realistic experimentation of market agents in this environment. Finally, our reinforcement learning market-makers deployed on this platform reveal important insights into the trade-offs involved in state-action space design and the learning dynamics of various RL architectures. This setup also allows for a meaningful comparative evaluation of market-makers with previous work. Collectively, these contributions provide a reproducible foundation for advancing market simulation frameworks and for developing and evaluating novel reinforcement learning approaches to the market-making problem.

More information

Identifiers

ORCID for Christopher Jaehoon Cho: orcid.org/0000-0003-2805-0352

ORCID for Tim Norman: orcid.org/0000-0002-6387-4034

ORCID for Manuel Nunes: orcid.org/0000-0002-7116-5502

Catalogue record

Export record

Altmetrics