Using High Density EMG to Proportionally Control 3D Model of Human Hand

Firman Isma Serdana, Silvia Muceli, Dario Farina

Abstract


Control of human hand using surface electromyography (EMG) is already established in various mechanisms, but proportionally controlling magnitudes degrees of freedom (DOF) of humanoid hand model is still highly developed in recent years. This paper proposes another method to achieve a proportional estimation and control of human’s hand multiple DOFs. Gestures in the form of American Sign Language (ABCDFIKLOW) were chosen as the targets, of which ten alphabetical gestures were specifically used following their clarity on its 3D model. Then the dataset of the movements gestures was simultaneously recorded using High-density electromyography (HD-EMG) and motion capture system. Sensor placements were on intrinsic - extrinsic muscles for HD-EMG and finger joints for the motion capture system. To derive the proportional control in time series between both datasets (HD-EMG and kinematics data), neural network (NN) and k-Nearest Neighbour were used. The models produced around 70-95 % (R index) accuracy for the eleven DOFs in four healthy subjects’ hand. kNN’s performance was better than NN, even if the input features were reduced either using manual selections or principal component analysis (PCA). The time series controls could also identify most sign language gestures (9 of 10), with difficulty was given on O gesture. The false interpretation was because of nearly identical muscle’s EMG and kinematics data between O and C. This paper intends to extend its conference version [1] by adding more in-depth Results and Discussion along making other sections more comprehensive.

Keywords


High-density electromyography; hand kinematics; neural network; k-nearest neighbor

Full Text:

PDF

References


F. I. Serdana, “Controlling 3D Model of Human Hand Exploiting Synergistic Activation of The Upper Limb Muscles,†IES 2022 - 2022 International Electronics Symposium: Energy Development for Climate Change Solution and Clean Energy Transition, Proceeding, pp. 142–149, 2022, doi: 10.1109/IES55876.2022.9888488.

A. Theuer et al., “Case Report: Optimizing Daily Function for People with Below-elbow Limb Deficiency with the SoftHand Pro.,†Open Journal of Occupational Therapy, vol. 8, no. 4, pp. 1–9, Sep. 2020, doi: 10.15453/2168-6408.1602.

M. H. Hasbani, D. Y. Barsakcioglu, M. K. Jung, and D. Farina, “Simultaneous and proportional control of wrist and hand degrees of freedom with kinematic prediction models from high-density EMG,†Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS, vol. 2022-July, pp. 764–767, 2022, doi: 10.1109/EMBC48229.2022.9871346.

M. Nowak, I. Vujaklija, A. Sturma, C. Castellini, and D. Farina, “Simultaneous and Proportional Real-Time Myocontrol of up to Three Degrees of Freedom of the Wrist and Hand,†IEEE Trans Biomed Eng, 2022, doi: 10.1109/TBME.2022.3194104.

“bebionic Hand EQD | The most lifelike prosthetic hand.†https://www.ottobock.com/en-us/product/8E70 (accessed Dec. 09, 2022).

“Open Bionics - Turning Disabilities into Superpowers.†https://openbionics.com/en/ (accessed Dec. 09, 2022).

M. N. Castro and S. Dosen, “Continuous Semi-autonomous Prosthesis Control Using a Depth Sensor on the Hand,†Front Neurorobot, vol. 16, p. 51, Mar. 2022, doi: 10.3389/FNBOT.2022.814973/BIBTEX.

P. Weiner, J. Starke, S. Rader, F. Hundhausen, and T. Asfour, “Designing Prosthetic Hands With Embodied Intelligence: The KIT Prosthetic Hands,†Front Neurorobot, vol. 16, p. 25, Mar. 2022, doi: 10.3389/FNBOT.2022.815716/BIBTEX.

R. F. Becker, “The cerebral cortex of man. By Wilder Penfield and Theodore Rasmussen. The Macmillan Company, New York, N.Y. 1950. 248 pp,†Am. J. Phys. Anthropol., vol. 11, no. 3, pp. 441–444, 1953.

B. Xu et al., “Natural grasping movement recognition and force estimation using electromyography,†Front Neurosci, vol. 16, p. 1020086, Oct. 2022, doi: 10.3389/FNINS.2022.1020086.

R. J. Smith, F. Tenore, D. Huberdeau, R. Etienne-cummings, and N. v Thakor, “Continuous Decoding of Finger Position from Surface EMG Signals for the Control of Powered Prostheses,†Crit. Rev., pp, pp. 197–200, 2009.

M. Hioki and H. Kawasaki, “Estimation of Finger Joint Angles from sEMG Using a Neural Network Including Time Delay Factor and Recurrent Structure,†ISRN Rehabil., vol. 2012, pp. 1–13, 2012.

J. Ngeo, T. Tamei, and T. Shibata, “Estimation of continuous multi-DOF finger joint kinematics from surface EMG using a multi-output Gaussian Process,†in 2014 36th Annu, 2014: Int. Conf. IEEE Eng. Med. Biol. Soc. EMBC, 2014, pp. 3537–3540.

C. Chen, G. Chai, W. Guo, X. Sheng, D. Farina, and X. Zhu, “Prediction of finger kinematics from discharge timings of motor units: Implications for intuitive control of myoelectric prostheses,†J. Neural Eng, vol. 16, p. 2, 2019.

D. Blana, W. Murray, A. Ganguly, A. Krasoulis, K. Nazarpour, and E. Chadwick, “Model-based control of individual finger movements for prosthetic hand function,†Keele Univ, pp. 1–9, 2019.

S. Muceli and D. Farina, “Simultaneous and proportional estimation of hand kinematics from EMG during mirrored movements at multiple degrees-of-freedom,†IEEE Trans. Neural Syst. Rehabil. Eng, vol. 20, no. 3, pp. 371–378, 2012.

W. Li, P. Shi, and H. Yu, “Gesture Recognition Using Surface Electromyography and Deep Learning for Prostheses Hand: State-of-the-Art, Challenges, and Future,†Front Neurosci, vol. 15, p. 259, Apr. 2021, doi: 10.3389/FNINS.2021.621885/BIBTEX.

N. J. Jarque-Bou, J. L. Sancho-Bru, and M. Vergara, “A systematic review of EMG applications for the characterization of forearm and hand muscle activity during activities of daily living: Results, challenges, and open issues,†Sensors, vol. 21, no. 9. MDPI AG, May 01, 2021. doi: 10.3390/s21093035.

C. Boudreau, J. Corkum, I. Grant, and D. T. Tang, “A comparative study using electromyography to assess hand exercises for rehabilitation after ulnar nerve decompression,†Journal of Plastic, Reconstructive & Aesthetic Surgery, vol. 75, no. 1, pp. 307–313, Jan. 2022, doi: 10.1016/J.BJPS.2021.08.042.

A. A. Adewuyi, L. J. Hargrove, and T. A. Kuiken, “An Analysis of Intrinsic and Extrinsic Hand Muscle EMG for Improved Pattern Recognition Control,†in IEEE Trans, vol. 24, no. 4: neural Syst. Rehabil. Eng. a Publ. IEEE Eng. Med. Biol. Soc, 2016, pp. 485–494.

K. Fujimura, H. Kagaya, and H. Tanikawa, “Kinematic Analysis for Repetitive Peripheral Magnetic Stimulation of the Intrinsic Muscles of the Hand,†Applied Sciences (Switzerland), vol. 12, no. 18, Sep. 2022, doi: 10.3390/app12189015.

M. Barsotti, S. Dupan, I. Vujaklija, S. Došen, A. Frisoli, and D. Farina, “Online Finger Control Using High-Density EMG and Minimal Training Data for Robotic Applications,†IEEE Robot Autom Lett, vol. 4, no. 2, pp. 217–223, Apr. 2019, doi: 10.1109/LRA.2018.2885753.

E. J. Weiss and M. Flanders, “Muscular and postural synergies of the human hand,†J. Neurophysiol., vol. 92, no. 1, pp. 523–535, 2004.

S. Tateno, H. Liu, and J. Ou, “Development of sign language motion recognition system for hearing-impaired people using electromyography signal,†Sensors (Switzerland), vol. 20, no. 20, pp. 1–22, Oct. 2020, doi: 10.3390/s20205807.

I. Carpinella, P. Mazzoleni, M. Rabuffetti, R. Thorsen, and M. Ferrarin, “Experimental protocol for the kinematic analysis of the hand: Definition and repeatability,†Gait Posture, vol. 23, no. 4, pp. 445–454, 2006.

T. Bao, A. Zaidi, S. Xie, and Z. Zhang, “Surface-EMG based Wrist Kinematics Estimation using Convolutional Neural Network,†p, pp. 1–4, 2019.

A. Sharma, P. Madhushri, V. Kushvaha, and A. Kumar, “Prediction of the Fracture Toughness of Silicafilled Epoxy Composites using K-Nearest Neighbor (KNN) Method,†2020 International Conference on Computational Performance Evaluation, ComPE 2020, pp. 194–198, Jul. 2020, doi: 10.1109/COMPE49325.2020.9200093.

U. Phutane, M. Roller, A. Boebel, and S. Leyendecker, “Optimal control of grasping problem using postural synergies,†in Advances in Transdisciplinary Engineering, Aug. 2020, vol. 11, pp. 206–213. doi: 10.3233/ATDE200026.

J. A. Raszewski, A. C. Black, and M. Varacallo, “Anatomy, Shoulder and Upper Limb, Hand Compartments,†StatPearls, Sep. 2022, Accessed: Dec. 09, 2022. [Online]. Available: https://www.ncbi.nlm.nih.gov/books/NBK532942/

N. J. Jarque-Bou, M. Vergara, J. L. Sancho-Bru, V. Gracia-Ibáñez, and A. Roda-Sales, “A calibrated database of kinematics and EMG of the forearm and hand during activities of daily living,†Sci Data, vol. 6, no. 1, Dec. 2019, doi: 10.1038/s41597-019-0285-1.

X. Hu, A. Song, J. Wang, H. Zeng, and W. Wei, “Finger Movement Recognition via High-Density Electromyography of Intrinsic and Extrinsic Hand Muscles,†Sci Data, vol. 9, no. 1, p. 373, 2022, doi: 10.1038/s41597-022-01484-2.

B.-S. Lin, I.-J. Lee, P.-Y. Chiang, S.-Y. Huang, and C.-W. Peng, “A Modular Data Glove System for Finger and Hand Motion Capture Based on Inertial Sensors,†in J, vol. 39, no. 4: Med. Biol. Eng, 2019, pp. 532–540.




DOI: http://dx.doi.org/10.18517/ijaseit.13.3.18380

Refbacks

  • There are currently no refbacks.


Published by INSIGHT - Indonesian Society for Knowledge and Human Development