Spatio-temporal visual learning for home-based monitoring
Peer reviewed, Journal article
Published version
![Thumbnail](/usn-xmlui/bitstream/handle/11250/3145859/1-s2.0-S1566253523003007-main.pdf.jpg?sequence=6&isAllowed=y)
View/ Open
Date
2024Metadata
Show full item recordCollections
- Institutt for mikrosystemer [563]
- Publikasjoner fra CRIStin [3727]
Original version
Djenouri, Y., Belbachir, A. N., Cano, A., & Belhadi, A. (2024). Spatio-temporal visual learning for home-based monitoring. Information Fusion, 101, Artikkel 101984. https://doi.org/10.1016/j.inffus.2023.101984Abstract
This paper introduces a novel concept for Home-based Monitoring (HM) that enables robust analysis and understanding of activities towards improved caring and safety. Spatio-Temporal Visual Learning for HM (STVL-HM) is a novel method that learns from sensor data that is jointly represented in space and time in order to robustify the HM process. We propose a hybrid model based on a Convolution Neural Network (CNN) and Transformers. The CNN first learns the visual spatial features from various sensor data. The learned visual features are then fed into the transformer, which captures temporal information by observing the sensor status at various timestamps. STVL-HM has been tested using Kinetics-400, the real use case of human activity recognition scenario for HM data. The results reveal the clear superiority of the STVL-HM compared to the recent baseline HM solutions.