The University of Southampton
University of Southampton Institutional Repository
Warning ePrints Soton is experiencing an issue with some file downloads not being available. We are working hard to fix this. Please bear with us.

AI3SD Video: Hyperparameter Optimisation for Graph Neural Networks

AI3SD Video: Hyperparameter Optimisation for Graph Neural Networks
AI3SD Video: Hyperparameter Optimisation for Graph Neural Networks
Traditional deep learning has made significant progress on various problems, from computer vision to natural language processing. For graph problems, there are still many challenges. Graph neural networks (GNNs) have been proposed for a wide range of learning tasks in the graph domain. In particular, in recent years, an increasing number of GNN models were applied to model molecular graphs and predict the properties of the corresponding molecules. However, a direct impediment to achieve good performance with the lower computational cost is to select appropriate hyperparameters. Meanwhile, many molecular datasets are far smaller than many other datasets in typical deep learning applications. Most hyperparameter optimization (HPO) methods for deep learning have not been explored in terms of their efficiencies on such small datasets in the molecular domain. We conducted theoretical analyses for popular HPO methods (random search, TPE, and CMA-ES) and proposed a genetic algorithm with hierarchical evaluation strategy and tree-structured mutation for HPO. Finally, we believe that our work will motivate further research to GNNs as applied to molecular machine learning problems and facilitate scientific discovery.
AI, AI3SD Event, Artificial Intelligence, Chemical Space, Chemistry, Machine Learning, ML, Neural Networks
Yuan, Yingfang (James)
4c17c1cf-7ade-45ed-a13c-b64a773ceb3d
Frey, Jeremy G.
ba60c559-c4af-44f1-87e6-ce69819bf23f
Kanza, Samantha
b73bcf34-3ff8-4691-bd09-aa657dcff420
Niranjan, Mahesan
5cbaeea8-7288-4b55-a89c-c43d212ddd4f
Yuan, Yingfang (James)
4c17c1cf-7ade-45ed-a13c-b64a773ceb3d
Frey, Jeremy G.
ba60c559-c4af-44f1-87e6-ce69819bf23f
Kanza, Samantha
b73bcf34-3ff8-4691-bd09-aa657dcff420
Niranjan, Mahesan
5cbaeea8-7288-4b55-a89c-c43d212ddd4f

Yuan, Yingfang (James) (2021) AI3SD Video: Hyperparameter Optimisation for Graph Neural Networks. Frey, Jeremy G., Kanza, Samantha and Niranjan, Mahesan (eds.) AI3SD Autumn Seminar Series 2021. 13 Oct - 15 Dec 2021. (doi:10.5258/SOTON/AI3SD0171).

Record type: Conference or Workshop Item (Other)

Abstract

Traditional deep learning has made significant progress on various problems, from computer vision to natural language processing. For graph problems, there are still many challenges. Graph neural networks (GNNs) have been proposed for a wide range of learning tasks in the graph domain. In particular, in recent years, an increasing number of GNN models were applied to model molecular graphs and predict the properties of the corresponding molecules. However, a direct impediment to achieve good performance with the lower computational cost is to select appropriate hyperparameters. Meanwhile, many molecular datasets are far smaller than many other datasets in typical deep learning applications. Most hyperparameter optimization (HPO) methods for deep learning have not been explored in terms of their efficiencies on such small datasets in the molecular domain. We conducted theoretical analyses for popular HPO methods (random search, TPE, and CMA-ES) and proposed a genetic algorithm with hierarchical evaluation strategy and tree-structured mutation for HPO. Finally, we believe that our work will motivate further research to GNNs as applied to molecular machine learning problems and facilitate scientific discovery.

Video
AI3SDAutumnSeminar-011221-YingfangYuan - Version of Record
Available under License Creative Commons Attribution.
Download (513MB)

More information

Published date: 1 December 2021
Additional Information: Yingfang (James) Yuan is a PhD candidate working under the supervision of Dr Wei Pang, Prof. Mike Chantler and Prof. George M. Coghill (external) in the School of Mathematical and Computer Sciences at Heriot-Watt University. Yingfang received his MSc in Big Data and High-Performance Computing at University of Liverpool. His research interests include Machine Learning; Deep Learning (especially in Graph Neural Networks and AutoML); He particularly focused on investigating the impact of hyperparameter optimization on graph neural networks applied to predict molecular properties and the efficiency of hyperparameter optimization approaches.
Venue - Dates: AI3SD Autumn Seminar Series 2021, 2021-10-13 - 2021-12-15
Keywords: AI, AI3SD Event, Artificial Intelligence, Chemical Space, Chemistry, Machine Learning, ML, Neural Networks

Identifiers

Local EPrints ID: 453342
URI: http://eprints.soton.ac.uk/id/eprint/453342
PURE UUID: b5876837-34cb-48eb-a0a3-184acd07eb2f
ORCID for Jeremy G. Frey: ORCID iD orcid.org/0000-0003-0842-4302
ORCID for Samantha Kanza: ORCID iD orcid.org/0000-0002-4831-9489
ORCID for Mahesan Niranjan: ORCID iD orcid.org/0000-0001-7021-140X

Catalogue record

Date deposited: 13 Jan 2022 17:46
Last modified: 14 Jan 2022 02:53

Export record

Altmetrics

Contributors

Author: Yingfang (James) Yuan
Editor: Jeremy G. Frey ORCID iD
Editor: Samantha Kanza ORCID iD
Editor: Mahesan Niranjan ORCID iD

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×