Spatio-temporal visual learning for home-based monitoring
Peer reviewed, Journal article
Published version
Permanent lenke
https://hdl.handle.net/11250/3145859Utgivelsesdato
2024Metadata
Vis full innførselSamlinger
- Institutt for mikrosystemer [563]
- Publikasjoner fra CRIStin [3727]
Originalversjon
Djenouri, Y., Belbachir, A. N., Cano, A., & Belhadi, A. (2024). Spatio-temporal visual learning for home-based monitoring. Information Fusion, 101, Artikkel 101984. https://doi.org/10.1016/j.inffus.2023.101984Sammendrag
This paper introduces a novel concept for Home-based Monitoring (HM) that enables robust analysis and understanding of activities towards improved caring and safety. Spatio-Temporal Visual Learning for HM (STVL-HM) is a novel method that learns from sensor data that is jointly represented in space and time in order to robustify the HM process. We propose a hybrid model based on a Convolution Neural Network (CNN) and Transformers. The CNN first learns the visual spatial features from various sensor data. The learned visual features are then fed into the transformer, which captures temporal information by observing the sensor status at various timestamps. STVL-HM has been tested using Kinetics-400, the real use case of human activity recognition scenario for HM data. The results reveal the clear superiority of the STVL-HM compared to the recent baseline HM solutions.