CFDComm: an optimized library for scalable point-to-point communication for general CFD applications

Haeri, Sina and Shrimpton, John S. (2012) CFDComm: an optimized library for scalable point-to-point communication for general CFD applications. In 2012 IEEE 14th International Conference on High Performance Computing and Communication & 2012 IEEE 9th International Conference on Embedded Software and Systems (HPCC-ICESS). IEEE. pp. 1001-1006 . (doi:10.1109/HPCC.2012.146).

Abstract

Domain decomposition is the most widely used technique to achieve parallelism in CFD applications. For complicated geometries usually graph partitioning programs are used to decompose the domain into smaller computational blocks such that the computation load is balanced and communication cost is minimized. In this paper an algorithm is provided and tested which avoids deadlocks in complicated communications patterns inherited from the graph decomposition process. The basic algorithm is implemented using FORTRAN 95 and MPI and then several optimization techniques are used to increase the scalability of the library which include addition of topologies, overlap of communication and computation to mask the message passing latency and non-blocking communication. The library is tested for up to 512 cores on the Iridis-3 cluster which incorporates 1008 compute nodes each composed of 2, 2.4 GHz 6-core Westmere processors. IO and inter-node communication is via a fast Infiniband network which is composed of groups of 32 nodes connected by DDR links to a 48 port QDR leaf-switch. The leaf switches then have 4 trunked QDR connections to 4 QDR 48-port core switches.

This record has no associated files available for download.

Contributors

Download statistics