Efficient teacher-student architectures for Human Activity Recognition via soft labels and binarization

Abstract

Human Activity Recognition (HAR) applications are most commonly deployed on embedded systems with limited computational resources. Our work focuses on applying deep learning methods to HAR and developing compact architectures. The first chapter of this report introduces a novel label representation for HAR in which we introduce the soft label shown to be capable of inducing better representation performance than that of the one-hot label. We will further investigate the teacher-student architecture for HAR. In our approach, we incorporate soft label by the teacher that supervises the next generation of training via the students. Experiments on 3 benchmark datasets, widely used in the community, which confirm that after a few generations of training, the model's performance surpasses that of the one-hot label. We also introduce the ECE, to avoid over-confident predictions, we use ECE as a performance metric, to evaluate the calibration performance of the HAR models. The experimental results also confirm that the teacher-student architecture effectively reduces the ECE and trains well-calibrated networks. In the second chapter of this report, we evaluate the application of Binary Neural Networks (BNNs) in Human Activity Recognition (HAR) more suitable for constraints of embedded systems the features of embedded systems. Our goal is to significantly reduce the storage requirements and forward propagation latency of the model. We use XNOR-Net as the backbone architecture, where the weights, activation functions, and inputs to the convolutional layer are binary. The most crucial aspect is that the convolution operation is replaced by XNOR, resulting in a 32-fold reduction in memory usage and a 58-fold reduction in convolution operation latency. This enables operations to be performed on CPUs with limited computing power, rather than powerful GPUs, in most cases. We also examine the impact of using BNNs on the model's performance and the potential for transfer learning. Our findings show that these benefits do not come at the cost of accuracy or Expected Calibration Error (ECE) performance. However, the dataset we used has different sensors in different body parts, making transfer learning challenging. In the third chapter, we study the application of a hybrid XNOR-Net and teacher-student architecture in HAR. The teacher network is first trained with a hard label that supervises the BNN student networks. Our approach improves the performance of future generations (i.e., the students of the student). Finally, as part of our previous research, we participated in the OU-ISIR Wearable Sensor-based Gait Challenge in 2019 as part of an international competition in HAR and finished as runners-up. This involved gender and age-related multitasking learning. The gradient normalization algorithm was used in conjunction with the hybrid ResNet and BLSTM blocks. However, we no longer use it in subsequent research as the employed dataset is a single classification challenge rather than multi-task learning. This research contributes to the ongoing advancement in HAR, offering insights and methodologies that may inspire future research in this field.

More information

Identifiers

ORCID for Kate Farrahi: orcid.org/0000-0001-6775-127X

ORCID for Mahesan Niranjan: orcid.org/0000-0001-7021-140X

Catalogue record

Export record