The University of Southampton
University of Southampton Institutional Repository

Chaotic methods for the strong scalability of CFD

Chaotic methods for the strong scalability of CFD
Chaotic methods for the strong scalability of CFD
Supercomputing power has been doubling approximately every 14 months for at least three decades, increasing the capabilities of scientific modelling at a similar rate. The first machines capable of one ExaFLOP (1018 floating-point operations per second) are expected by 2020. However, architectural changes required to reach ‘exascale’ are significant, with energy efficiency constraints leading to a huge growth in parallelization. A new era of computing has arrived, dubbed the ‘many-core’ era, in which the number of computing cores is increasing faster than CFD simulation sizes – prompting the research question for this thesis:

‘What limits the strong scalability of CFD and its ability to handle many-core architectures? What can be done to improve the CFD algorithms in this respect?’

A number of scalability investigations have been performed from 1 through to 2048 cores, using a semi-implicit, finite-volume CFD code: ReFRESCO; and the University of Southampton supercomputer: Iridis4. The main bottleneck to strong scalability is shown to be the linear equation-system solvers, occupying up to 95% of total wall-time on 2048 cores – where the poor scalability arises from synchronous, global, inter-process communications. Experiments have been performed with alternative, state-of-the-art linear solvers and preconditioners, without significant improvements, which motivates novel research into scalable linear solvers for CFD.

The theory of ‘chaotic relaxation’ has been used to create a completely asynchronous Jacobi-like ‘chaotic solver’, showing almost perfect scalability, and performance far greater than their synchronous counterparts. However, these solvers lack the absolute numerical power to compete with existing solvers, especially as the resolution of the simulations increases. Following this, chaotic relaxation theory has been used to create a novel ‘chaotic-cycle’ multigrid solver, combining aspects of the chaotic solver and classical multigrid methods.

Both of the solvers have been verified and tested using canonical test cases and practical CFD simulations. On 2048 cores, the chaotic-cycle multigrid solver performs up to 7.7× faster than a typical Krylov Subspace solver and 13.3× faster than classical V-cycle multigrid. With improvements to the implementation of coarse-grid communications and desynchronized residual computations, it is likely that the chaotic-cycle multigrid method will continue scaling to many thousands of cores, thus removing the main bottleneck to the strong-scalability of CFD.

The novel chaotic solver and chaotic-cycle multigrid methods have been implemented as an open-source library, Chaos. It is hoped that work on these scalable solvers can be continued and applied to other disciplines.
University of Southampton
Hawkes, James
3882428e-838b-4e54-9fbf-ba85ba080dc9
Hawkes, James
3882428e-838b-4e54-9fbf-ba85ba080dc9
Turnock, Stephen
d6442f5c-d9af-4fdb-8406-7c79a92b26ce

Hawkes, James (2017) Chaotic methods for the strong scalability of CFD. University of Southampton, Doctoral Thesis, 214pp.

Record type: Thesis (Doctoral)

Abstract

Supercomputing power has been doubling approximately every 14 months for at least three decades, increasing the capabilities of scientific modelling at a similar rate. The first machines capable of one ExaFLOP (1018 floating-point operations per second) are expected by 2020. However, architectural changes required to reach ‘exascale’ are significant, with energy efficiency constraints leading to a huge growth in parallelization. A new era of computing has arrived, dubbed the ‘many-core’ era, in which the number of computing cores is increasing faster than CFD simulation sizes – prompting the research question for this thesis:

‘What limits the strong scalability of CFD and its ability to handle many-core architectures? What can be done to improve the CFD algorithms in this respect?’

A number of scalability investigations have been performed from 1 through to 2048 cores, using a semi-implicit, finite-volume CFD code: ReFRESCO; and the University of Southampton supercomputer: Iridis4. The main bottleneck to strong scalability is shown to be the linear equation-system solvers, occupying up to 95% of total wall-time on 2048 cores – where the poor scalability arises from synchronous, global, inter-process communications. Experiments have been performed with alternative, state-of-the-art linear solvers and preconditioners, without significant improvements, which motivates novel research into scalable linear solvers for CFD.

The theory of ‘chaotic relaxation’ has been used to create a completely asynchronous Jacobi-like ‘chaotic solver’, showing almost perfect scalability, and performance far greater than their synchronous counterparts. However, these solvers lack the absolute numerical power to compete with existing solvers, especially as the resolution of the simulations increases. Following this, chaotic relaxation theory has been used to create a novel ‘chaotic-cycle’ multigrid solver, combining aspects of the chaotic solver and classical multigrid methods.

Both of the solvers have been verified and tested using canonical test cases and practical CFD simulations. On 2048 cores, the chaotic-cycle multigrid solver performs up to 7.7× faster than a typical Krylov Subspace solver and 13.3× faster than classical V-cycle multigrid. With improvements to the implementation of coarse-grid communications and desynchronized residual computations, it is likely that the chaotic-cycle multigrid method will continue scaling to many thousands of cores, thus removing the main bottleneck to the strong-scalability of CFD.

The novel chaotic solver and chaotic-cycle multigrid methods have been implemented as an open-source library, Chaos. It is hoped that work on these scalable solvers can be continued and applied to other disciplines.

Text
e-thesis HAWKES - Version of Record
Available under License University of Southampton Thesis Licence.
Download (37MB)

More information

Published date: August 2017

Identifiers

Local EPrints ID: 429492
URI: http://eprints.soton.ac.uk/id/eprint/429492
PURE UUID: 3e53936c-9d5a-40a0-8cd4-3a87ade46b22
ORCID for Stephen Turnock: ORCID iD orcid.org/0000-0001-6288-0400

Catalogue record

Date deposited: 27 Mar 2019 17:30
Last modified: 28 Mar 2019 01:37

Export record

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×