Engagement Level Detection Using Facial Extraction and Multi-Stacked Convolutional Neural Network in e-Learning Settings

Engagement in learning is crucial in maintaining student eagerness as fuel for student learning to achieve learning success. In e-Learning settings, engagement also plays an im-portant role as a factor influencing student learning success as students are regulating themselves during learning one-on-one using computer devices. Failing to detect student engagement may result in unsuccessful learning. In extreme cases, students may disengage and learning purpose cannot be achieved. This research aims to detect user engagement in e-Learning settings by using a deep learning approach with a feature extraction method. The dataset used in this study is the DAiSEE dataset. The dataset is in the form of videos and is extracted into frames and specific facial features. The facial area was extracted using the Mediapipe Library. The features include 1434 3D landmark features, 52 action unit features, one depth estimation feature, and three head pose estimation features. This research also conducts image selection, which selecting frames with an interval of 15 in the train, validation, and test folders. The multi-stacked Convolutional Neural Network model was used to classify four levels of engagements. The results were compared to the existing approach in engagement detection. The untuned proposed model achieved an accuracy of 48.95%, while the tuned proposed model achieved an accuracy of 52.94%. The proposed model is able to compete with state-of-the-art methods using a less complex model. Even so, the model still cannot learn labels perfectly due to a very imbalanced dataset and a possibility of model overfitting. This research is expected to be useful for educational institutions and the e-Learning field in the future.