Recognition of Emotion in Indian Classical Dance Using EMG Signal

Shraddha A. Mithbavkar, Milind S. Shah

Abstract


Indian classical dance forms like Kathak are an enrichment of Indian culture and tradition. These dance forms glorify its beauty by expressing nine emotions (Navras) such as Adbhut (amazed), Bhayanaka (fearful), Hasya (humorous), Karuna (tragic), Raudra (fierce), Shringar (loving smile), Veer (heroic), Bibhatsa (disgusted), and Shant (peaceful). Identifying correct emotions is an important task. The objective of this research work is to recognize Navras in the Kathak dance. Proposed research work can assist dance teachers in an accurate and unbiased evaluation process of dance examination. This research work analyzed the Electromyogram (EMG) signals acquired from eleven subjects. The EMG signals collected from the various locations on the face and neck represent the emotions and head movement. The EMG signals are processed to extract integrated EMG (IEMG) features. This research introduced a new feature named 'difference in IEMG feature' for improving the accuracy of emotion recognition. For the classification of nine emotions, the Least Square Support Vector Machine (LSSVM), Nonlinear Autoregressive Exogenous Network (NARX), and Long- and Short-Term Memory (LSTM) classifiers were used. The classifiers' performance is judged with head motion and without head motion. The classification accuracies are calculated using a maximum, variance, and mean of the feature. LSSVM, NARX, and LSTM classifiers achieved 60.80%, 81.67%, and 92.28% classification accuracies, respectively, using the IEMG feature and head motion. Using the new feature, LSSVM, NARX, and LSTM classifiers achieved 64.29%, 81.27%, and 93.63% classification accuracies, respectively. The overall classification accuracy improved by 1.46% by using the new feature.

Keywords


Emotion recognition; EMG; classifier; Indian classical dance; Kathak; Navras.

Full Text:

PDF

References


M. Ghosh, Natyashastra, Translation of Bharat Muni Sanskrit book, vol. 1. Calcutta: The Royal Asiatic Society of Bengal, 1950.

P. K. Srimani and T. Hegde, “Analysis of facial expressions with respect to Navras as in Bharathanatym styles using image processing,†Int. J. Knowl. Eng., vol. 3, no. 2, pp. 193–196, Nov. 2012.

P. V. V. Kishore, K. V. V. Kumar, and E. K. Kumar, “Indian Classical Dance Action Identification and Classification with Convolution Neural Networks,†Hindawi Adv. Multimed., vol. 2018, no. 5141402, pp. 1–10, Jan. 2018.

A. Mohanty and R. R. Sahay, “Rrasabodha: understanding Indian classical dance by recognizing emotions using deep learning,†Pattern Recognit., vol. 79, pp. 97–113, Jul. 2018.

A. Raheel, M. Majid, M. Alnowami, and S. M. Anwar, “Physiological Sensors Based Emotion Recognition While Experiencing Tactile Enhanced Multimedia,†Sensors, vol. 20, no. 14, Jul. 2020, doi: 10.3390/s20144037.

B. I. Jeon, B. J. Kang, H. C. Cho, and J. Kim, “Motion Recognition and an Accuracy Comparison of Left and Right Arms by EEG Signal Analysis,†Appl. Sci. J., vol. 9, no. 4885, pp. 1–16, 2019.

B. Rodriguez-Tapia, I. Soto, D. M. Martinez, and N. C. Arballo, “Myoelectric Interfaces and Related Applications: Current State of EMG Signal Processing–A Systematic Review,†IEEE Access, vol. 8, 2020, doi: 10.1109/ACCESS.2019.2963881.

T. Partala, V. Surakka, and T. Vanhala, “Real-time estimation of emotional experiences from facial expressions,†Interact. Comput., vol. 18, no. 2, pp. 208–226, 2006.

R. W. Picard, E. Vyzas, and J. Healey, “Toward machine emotional intelligence: analysis of the affective physiological state,†IEEE Trans. Pattern Anal. Mach. Intell., vol. 23, no. 10, pp. 1175–1191, Oct. 2001.

B. Cheng and G.-Y. Liu, “Emotion recognition from surface EMG signal using wavelet transform and neural network,†J. Comput. Appl., vol. 28, no. 2, Feb. 2008, doi: 10.3724/SP.J.1087.2008.00333.

Y. G. Yang and S. Yang, “Study of Emotion Recognition Based on Surface Electromyography and Improved Least Squares Support Vector Machine,†J. Comput., vol. 6, no. 8, Aug. 2011, doi: 10.4304/jcp.6.8.1707-1714.

S. Jerritta, M. Murugappan, K. Wan, and S. Yaacob, “Emotion recognition from facial EMG signals using higher order statistics and principal component analysis,†J. Chinese Inst. Eng., vol. 37, no. 3, Apr. 2014, doi: 10.1080/02533839.2013.799946.

V. Kehri, R. Ingle, S. Patil, and R. N. Awale, “Analysis of Facial EMG Signal for Emotion Recognition Using Wavelet Packet Transform and SVM,†M. Tanveer R. B. Pachori (eds.), Mach. Intell. Signal Anal. Adv. Intell. Syst. Comput., vol. 748, pp. 247–257, 2019.

S. A. Mithbavkar and M. S. Shah, “EMG based emotion recognition in Indian classical dance,†Biosci. Biotechnol. Res. Commun., vol. 13, no. 14, pp. 330–334, Dec. 2020.

S. A. Mithbavkar and M. S. Shah, “Recognition of Emotion Through Facial Expressions Using EMG Signal,†in International Conference on Nascent Technologies in Engineering (ICNTE), 2019, pp. 1–6.

L. Kulke, D. Feyerabend, and A. Schacht, “A Comparison of the Affectiva iMotions Facial Expression Analysis Software With EMG for Identifying Facial Expressions of Emotion,†Front. Psychol., vol. 11, Feb. 2020, doi: 10.3389/fpsyg.2020.00329.

M. O’Sullivan, A. Temko, A. Bocchino, C. O’Mahony, G. Boylan, and E. Popovici, “Analysis of a Low-Cost EEG Monitoring System and Dry Electrodes toward Clinical Use in the Neonatal ICU,†Sensors, vol. 19, no. 11, Jun. 2019, doi: 10.3390/s19112637.

T. S. H. Wingenbach, M. Brosnan, M. C. Pfaltz, P. Peyk, and C. Ashwin, “Perception of Discrete Emotions in Others: Evidence for Distinct Facial Mimicry Patterns,†Sci. Rep., vol. 10, no. 1, Dec. 2020, doi: 10.1038/s41598-020-61563-5.

Y. Chen, Z. Yang, and J. Wang, “Eyebrow emotional expression recognition using surface EMG signals,†Neurocomputing, vol. 168, Nov. 2015, doi: 10.1016/j.neucom.2015.05.037.

X. Zhang, C. Xu, W. Xue, J. Hu, Y. He, and M. Gao, “Emotion Recognition Based on Multichannel Physiological Signals with Comprehensive Nonlinear Processing,†Sensors, vol. 18, no. 11, Nov. 2018, doi: 10.3390/s18113886.

B. D. Luciani, D. M. Desmet, A. A. Alkayyali, J. M. Leonardis, and D. B. Lipps, “Identifying the mechanical and neural properties of the sternocleidomastoid muscles,†J. Appl. Physiol., vol. 124, no. 5, May 2018, doi: 10.1152/japplphysiol.00892.2017.

C. F. Tan and W. Chen, “The relationship of head rotation angle and SCM EMG value for the development of AnS2,†2010.

A. Phinyomark, R. N. Khushaba, and E. Scheme, “Feature Extraction and Selection for Myoelectric Control Based on Wearable EMG Sensors,†Sensors, vol. 18, no. 5, May 2018, doi: 10.3390/s18051615.

M. Hamedi, S.-H. Salleh, C.-M. Ting, M. Astaraki, and A. M. Noor, “Robust Facial Expression Recognition for MuCI: A Comprehensive Neuromuscular Signal Analysis,†IEEE Trans. Affect. Comput., vol. 9, no. 1, Jan. 2018, doi: 10.1109/TAFFC.2016.2569098.

N. Nazmi et al., “Assessment on Stationarity of EMG Signals with Different Windows Size During Isotonic Contractions,†Appl. Sci., vol. 7, no. 10, Oct. 2017, doi: 10.3390/app7101050.

C. Spiewak, “A Comprehensive Study on EMG Feature Extraction and Classifiers,†Open Access J. Biomed. Eng. Biosci., vol. 1, no. 1, Feb. 2018, doi: 10.32474/OAJBEB.2018.01.000104.

U. Kaimkhani, B. Naz, and S. Narejo, “Rainfall Prediction Using Time Series Nonlinear Autoregressive Neural Network,†Int. J. Comput. Sci. Eng., vol. 8, no. 1, Jan. 2021, doi: 10.14445/23488387/IJCSE-V8I1P106.

Z. Boussaada, O. Curea, A. Remaci, H. Camblong, and N. Mrabet Bellaaj, “A Nonlinear Autoregressive Exogenous (NARX) Neural Network Model for the Prediction of the Daily Direct Solar Radiation,†Energies, vol. 11, no. 3, Mar. 2018, doi: 10.3390/en11030620.

J. Bilski, B. Kowalczyk, A. Marchlewska, and J. M. Zurada, “Local Levenberg-Marquardt Algorithm for Learning Feedforwad Neural Networks,†J. Artif. Intell. Soft Comput. Res., vol. 10, no. 4, Oct. 2020, doi: 10.2478/jaiscr-2020-0020.

G. Kłosowski, T. Rymarczyk, D. Wójcik, S. Skowron, T. Cieplak, and P. Adamkiewicz, “The Use of Time-Frequency Moments as Inputs of LSTM Network for ECG Signal Classification,†Electronics, vol. 9, no. 9, Sep. 2020, doi: 10.3390/electronics9091452.

J. Kumar, R. Goomer, and A. K. Singh, “Long Short Term Memory Recurrent Neural Network (LSTM-RNN) Based Workload Forecasting Model For Cloud Datacenters,†Procedia Comput. Sci., vol. 125, 2018, doi: 10.1016/j.procs.2017.12.087.

M. A. Bashar, R. Nayak, and N. Suzor, “Regularising LSTM classifier by transfer learning for detecting misogynistic tweets with small training set,†Knowl. Inf. Syst., vol. 62, no. 10, Oct. 2020, doi: 10.1007/s10115-020-01481-0.

D. Brzezinski and J. Stefanowski, “Prequential AUC: properties of the area under the ROC curve for data streams with concept drift,†Knowl. Inf. Syst., vol. 52, no. 2, Aug. 2017, doi: 10.1007/s10115-017-1022-8.

J.-M. Vivo, M. Franco, and D. Vicari, “Rethinking a ROC partial area index for evaluating the classification performance at a high specificity range,†Adv. Data Anal. Classif., vol. 12, no. 3, Sep. 2018, doi: 10.1007/s11634-017-0295-9.

Y. Ma, X. Liang, G. Sheng, J. T. Kwok, M. Wang, and G. Li, “Noniterative Sparse LS-SVM Based on Globally Representative Point Selection,†IEEE Trans. Neural Networks Learn. Syst., vol. 32, no. 2, Feb. 2021, doi: 10.1109/TNNLS.2020.2979466.




DOI: http://dx.doi.org/10.18517/ijaseit.11.4.14034

Refbacks

  • There are currently no refbacks.


Published by INSIGHT - Indonesian Society for Knowledge and Human Development