AI3SD Video: Hyperparameter optimisation for graph neural networks

Abstract

Traditional deep learning has made significant progress on various problems, from computer vision to natural language processing. For graph problems, there are still many challenges. Graph neural networks (GNNs) have been proposed for a wide range of learning tasks in the graph domain. In particular, in recent years, an increasing number of GNN models were applied to model molecular graphs and predict the properties of the corresponding molecules. However, a direct impediment to achieve good performance with the lower computational cost is to select appropriate hyperparameters. Meanwhile, many molecular datasets are far smaller than many other datasets in typical deep learning applications. Most hyperparameter optimization (HPO) methods for deep learning have not been explored in terms of their efficiencies on such small datasets in the molecular domain. We conducted theoretical analyses for popular HPO methods (random search, TPE, and CMA-ES) and proposed a genetic algorithm with hierarchical evaluation strategy and tree-structured mutation for HPO. Finally, we believe that our work will motivate further research to GNNs as applied to molecular machine learning problems and facilitate scientific discovery.

More information

Identifiers

ORCID for Jeremy G. Frey: orcid.org/0000-0003-0842-4302

ORCID for Samantha Kanza: orcid.org/0000-0002-4831-9489

ORCID for Mahesan Niranjan: orcid.org/0000-0001-7021-140X

Catalogue record

Export record

Altmetrics