Enhancing Human Activity Recognition with Language Analysis and BERT

Abstract

Recognizing activities within smart home environments is imperative for developing automated services catering to inhabitants’ needs. However, this task presents significant challenges due to the variability of environments, diverse sensorymotor systems, user behavior patterns, signal sparsity, and model redundancy. End-to-end systems often struggle to automatically extract essential features and require access to context and domain knowledge. To address the feature extraction challenges in activity recognition within smart homes, we propose an innovative approach that integrates methods from both Natural Language Processing (NLP) and Time Series Classification (TSC) domains. The efficacy of our proposed method is evaluated using aruba dataset sourced from the Center for Advanced Studies in Adaptive Systems (CASAS). WordPiece tokenizer is used to tokenize the words in the sentences. Later, the tokenized sentences are embedded, trained and classified using Fine-tuned Bidirectional Encoder Representations from Transformers (BERT) model for sequence classification. Our method demonstrates promising results in offline activity classification