Memristor-based Spiking Neural Networks

Abstract

Emerging non-volatile memory devices, known as memristors, have demonstrated remarkable perspective in neuromorphic hardware designs, particularly in spiking neural network (SNNs) hardware implementation. Memristor-based SNNs have been applied in solving tasks (e.g. image classification and pattern recognition) traditionally solved by conventional artificial neural networks (ANNs), and more attempts in varying disciplines are still being made to exploit the potential of this new research topic. To apply memristors in neuromorphic applications (strictly defined as applications using SNNs in this thesis), two pathways can be followed. One starts by characterising and controlling memristor devices by utilising hardware infrastructure, which is later mapped with the application's higher-level functions (e.g. matrix multiplications). Another embeds data-driven memristor models in software simulators to emulate the application with parameters extracted from real devices.

This thesis aims to build a cohesive pipeline for bringing memristor-based SNNs to practical use following these two pathways. To achieve this goal, three key designs have been developed. The first one is an FPGA-based digital interface that is part of a memristor characterisation and control system which enables 64-channel parallel read/write operations and high-speed data processing. The control system, developed by the author and two other researchers, not only acts as a testing tool for collecting memristor characteristics but also delivers higher-level functions with memristor arrays in neuromorphic designs. Whilst the thesis focuses on a usage scenario for memristors, this system includes more powerful, versatile testing functionality for other two-terminal emerging memory devices. The FPGA-based interface was developed by the author solely, and it achieves 64-channel level parallelism in the application aspect and complex digital system design and organisation in the engineering aspect. The digital interface is validated by resistor handling and current-voltage sweep experiments.

The second design is the first Python-based algorithm-level simulator, NeuroPack, for memristor-based SNNs with a data-driven memristor model. NeuroPack aims to allow users to quickly validate a neuromorphic concept in the pre-hardware design phase. This tool provides a wide range of optional neuron, plasticity, and device models for users to choose from and answers a fundamental question: is the design functional given the knowledge of the memristor switching dynamics, assuming that the rest of the design is functionally perfect? If yes, the design can move ahead towards the next step. Besides, NeuroPack stores internal variables, including membrane voltages, neuron firing history, and memristor states. With a built-in analysis tool, users can analyse and visualise inference results, observe the evolution of weights and membrane voltages and monitor memristor behaviours. A handwritten digit recognition task in the MNIST dataset showcased how NeuroPack assists users in confirming system validation and exploring sensitivity to critical design choices.

The third design tries to expand the usage of memristor-based SNNs to high-dimensional large-scale applications. To do so, a bespoke simulation framework extended from the second design is developed. The first sentiment analysis task in the IMDB movie reviews dataset is exhibited with this framework. Two paths are taken to train spiking neural networks with memristor models: 1) by converting a pre-trained artificial neural network (ANN) to a memristor-based SNN, or 2) by directly training a memristor-based SNN. These two paths have two application scenarios: offline classification and online training. By converting a pre-trained ANN to a memristor-based SNN and training the memristor-based SNN directly, we achieve a classification accuracy of 85.88% and 84.86%, respectively, with the equivalent ANN achieving a baseline training accuracy of 86.02%. From ANNs to SNNs and from non-memristive synapses to data-driven memristive synapses, comparable classification accuracy can be achieved in simulation. In addition, investigations of the neural network sensitivity to global parameters such as spike train length, the read noise, and the weight updating stop conditions have also been given. These investigations further suggest that the simulation framework with statistic memristor models that use experimental data for statistical fitting taking the two paths presented in this chapter can help to exploit the potential of incorporating memristor-based SNNs in text classification tasks.

In summary, with the aid of the designs presented in this thesis, we envisage the two pathways now are complete to achieve neuromorphic applications with memristors, especially in performing text classification tasks. The thesis is concluded with the achieved contributions and future perspectives toward AI hardware.

More information

Identifiers

ORCID for Themis Prodromakis: orcid.org/0000-0002-6267-6909

Catalogue record

Export record