Mental Health State Classification Using Facial Emotion Recognition and Detection

Adel Aref Ali Al-zanam, Omer Hussein Abdou Elsayed Hussein Alhomery, Choo Peng Tan

Abstract


Analyzing and understanding emotion can help in various aspects, such as realizing one’s attitude, behavior, etc. By understanding one’s emotions, one's mental health state can be calculated, which can help in the medical field by classifying whether one is mentally stable or not. Facial Recognition is one of the many fields of computer vision that utilizes convolutional networks or Conv Nets to perform, train, and learn. Conv Nets and other machine learning algorithms have evolved to adapt better to larger datasets. One of the advancements in Conv Nets and machines is the introduction of various Conv architectures like VGGNet. Thus, this study will present a mental health state classification approach based on facial emotion recognition. The methodology comprises several interconnected components, including preprocessing, feature extraction using Principal Component Analysis (PCA) and VGGNet, and classification using Support Vector Machines (SVM) and Multilayer Perceptron (MLP). The FER2013 dataset tests multiple models’ performances, and the best model is employed in the mental health state classification. The best model, which combines Visual Geometry Group Network (VGGNet) feature extraction with SVM classification, achieved an accuracy of 66%, demonstrating the effectiveness of the proposed methodology. By leveraging facial emotion recognition and machine learning techniques, the study aims to develop an effective method.

Keywords


Principal component analysis; multilayer perceptron; facial emotion recognition; CNN; visual geometry group

Full Text:

PDF

References


Tivatansakul, S. O. (2014). Emotional healthcare system: Emotion detection by facial expressions using Japanese database. In 2014 6th computer science and electronic engineering conference (CEEC), 41- 46. doi: 10.1109/CEEC.2014.6958552

M. Smith, R. S., “Improving Emotional Health,†May 15, 2014 [Online]. Available: http://www.helpguide.org/mental/mental_emotional_health.html. [Assessed Jan. 5, 2023]

Elliott, E. A. (2013). Facial expressions, emotions, and sign languages. Frontiers in Psychology, 115. doi: 10.3389/fpsyg.2013.00115

Garcia-Garcia, J. M. (2017). Emotion detection: a technology review. In Proceedings of the XVIII international conference on human computer interaction, 1-8.

doi: 10.1145/3123818.3123852

Du, S. T. (2014). Compound facial expressions of emotion. Proceedings of the national academy of sciences, E1454-E1462.

doi: 10.1073/pnas.1322355111

Chen, X. S. (2021). Better than humans: a method for inferring consumer shopping intentions by reading facial expressions. 2021 14th International Symposium on Computational Intelligence and Design, 142-147. doi: 10.1109/ISCID52796.2021.00041

Seidel, E.-M. H. (2010). The impact of facial emotional expressions on behavioral tendencies in females and males. Journal of Experimental Psychology: Human Perception and Performance, 500. doi: https://doi.org/10.1037/a0018169

Back, E. &. (2014). Revealing Variations in Perception of Mental States from Dynamic Facial Expressions: A Cautionary Note. PLoS ON, e84395. doi: 10.1371/journal.pone.0084395

Sun, F. W. (2010). Computer vision vs. human vision. In 9th IEEE International Conference on Cognitive Informatics, 3-3

doi: 10.1109/coginf.2010.5599750

Sathya, R. &. (2013). Comparison of Supervised and Unsupervised Learning Algorithms for Pattern Classification. International Journal of Advanced Research in Artificial Intelligence, 34-38.

doi: 10.14569/ijarai.2013.020206

Pandey, P. D. (2017). Enhancing the quality of satellite images by preprocessing and contrast enhancement. In Proceedings of the 2017 IEEE International Conference on Communication and Signal Processing, 0056-0060. doi: 10.1109/ICCSP.2017.8286525

Huang, M. Z. (2020). Preprocessing Method Flow of Under-screen Fingerprint Image. In Proceedings of the International Conference on Anti-Counterfeiting, Security and Identification, 49-53.

doi: 10.1109/ASID50160.2020.9271693

Priyanka C. Dighe, S. K. (2014). Survey on Image Resizing Techniques. International Journal of Science and Research (IJSR), 1444–1448.ISSN: 2319-7064

Surasak, T. T. (2018). Histogram of oriented gradients for human detection in video. Proceedings of 2018 5th International Conference on Business and Industrial Research: Smart Technology for Next Generation of Information, Engineering, Business and Social Science, 172–176. doi: 10.1109/ICBIR.2018.8391187

Zhang, L. Z. (2020). Histogram of Oriented Gradients Feature Extraction Without Normalization. In 2020 IEEE Asia Pacific conference on circuits and systems (APCCAS), 252-255. doi: 10.1109/APCCAS50809.2020.9301715

Kolahdouzi, M. S.-M. (2021). Face Trees for Expression Recognition. In 2021 16th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2021), 1 - 5. doi: 10.1109/FG52635.2021.9666986

Zhao, H. J. (2021). Bayesian based Facial Expression Recognition Transformer Model in Uncertainty. 2021 International Conference on Digital Society and Intelligent Systems (DSInS), 157-161

doi: 10.1109/DSInS54396.2021.9670628

Li, J. Z. (2009). Face Recognition System Using SVM Classifier and Feature Extraction by PCA and LDA Combination. In 2009 International Conference on Computational Intelligence and Software Engineering., 1-4. doi: 10.1109/CISE.2009.5364125

Fabiyi, S. D. (2019). Comparative Study of PCA and LDA for Rice Seeds Quality Inspection. IEEE AFRICON Conference, 0–3.

doi: 10.1109/AFRICON46755.2019.9134059

Ayeche, F. &. (2021). Face Recognition and Facial Expression Recognition Algorithm Based on Novel Features Extraction Descriptors HDG and HDGG. 2021 International Conference on Information Systems and Advanced Technologies (ICISAT), 1-5. doi: 10.1109/ICISAT54145.2021.9678425

Ahmad, M. S. (2022). Hyperspectral Image Classification - Traditional to Deep Models: A Survey for Future Prospects. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 968–999. doi: 10.1109/JSTARS.2021.3133021

Gill, R. &. (2021). A Deep Learning Approach for Real Time Facial Emotion Recognition. In 2021 10th International Conference on System Modeling & Advancement in Research Trends (SMART), 497-501. doi: 10.1109/SMART52563.2021.9676202

Sultana, A. K. (2021). A Deep CNN based Kaggle Contest Winning Model to Recognize Real-Time Facial Expression. 2021 5th International Conference on Electrical Engineering and Information & Communication Technology (ICEEICT), 1-5. doi: 10.1109/ICEEICT53905.2021.9667824

Tegani, S. &. (2021). Using COVID-19 Masks Dataset to Implement Deep Convolutional Neural Networks For Facial Emotion Recognition. In 2021 4th International Symposium on Advanced Electrical and Communication Technologies (ISAECT), 1-5. doi: 10.1109/ISAECT53699.2021.9668345

Chang C. Y.Lin Y. M.Zheng J. Y. Proceedings of the 2012 15th International Conference on Network-Based Information Systems, NBIS 2012 (2012), 79–87. doi:10.1016/j.neucom.2013.02.041

Lim, Y., Ng, K. W., Naveen, P., & Haw, S. C. (2022). Emotion Recognition by Facial Expression and Voice: Review and Analysis. Journal of Informatics and Web Engineering, 1(2), 45-54. doi:10.33093/jiwe.2022.1.2.4

Agrawal, A., & Mittal, ·. N. (2020). Using CNN for facial expression recognition: a study of the effects of kernel size and number of filters on accuracy. The Visual Computer, 36 (2), 405-412. doi:10.1007/s00371-019-01630-9

Simcock, G. M. (2020). Associations between Facial Emotion Recognition and Mental Health in Early Adolescence. International Journal of Environmental Research and Public Health, 330.

doi: 10.3390/ijerph17010330

Ashish Kumar Srivastav, D. V., “Interpolation (Definition, Formula) | Calculation with Examples,†March 26, 2023, Available: https://www.wallstreetmojo.com/interpolation/. [Assessed April 23, 2023]

Hastie, T., Montanari, A., Rosset, S., & Tibshirani, R. J. (2022). Surprises in High-dimensional Ridgeless Least Squares Interpolation. Annals of Statistics, 50(2), 949. doi: 10.1214/21-AOS2133




DOI: http://dx.doi.org/10.18517/ijaseit.13.6.19055

Refbacks

  • There are currently no refbacks.


Published by INSIGHT - Indonesian Society for Knowledge and Human Development