Deep Multimodal CNN Fusion Scheme for Accurate Stress Identification

Abstract

Stress is a major issue in today’s life. It harms health and lower work output. People sometimes do not notice when under stress. That is why early stress detection is important. This paper uses two types of body signals: ECG (Electrocardiogram) and EDA (Electrodermal Activity). Both are physiological signals and help measure stress levels. A deep learning model named CNN (Convolutional Neural Network) is used. CNN has many layers. Each layer captures different kinds of features—low-level, mid-level and high-level. These features are useful in identifying stress. Instead of using features from only one level, this paper combines all three levels. This process is named as hierarchical feature fusion. It helps in creating a strong and rich representation of the signals. The features from ECG and EDA are first extracted at different CNN layers. Then, a module termed MMTM (Multimodal Transfer Module) is used. This module helps combine features from both signals. It improves the way the model learns from the data. The model is tested using both raw data and features from selected frequency bands. Results show that using features from all three CNN levels gives better performance. The proposed model performs better than existing models when using frequency band features. This shows that combining low, mid and high-level CNN features with multimodal fusion is helpful. It improves the accuracy and generalization of stress detection. This method works better across different datasets and different people. The proposed system is a useful tool in real-world stress detection systems.