Electroencephalography Signal Analysis for Virtual Reality Sickness: Head-mounted Display and Screen-based

Galih Restu Fardian Suwandi, Siti Nurul Khotimah, Freddy Haryanto, - Suprijadi

Abstract


The use of virtual reality (VR) technology is growing in the current era of the COVID-19 pandemic. However, the use of VR is causing problems for its users. Some symptoms, such as nausea, headache, and eye strain, are felt after using VR. These symptoms are called VR sickness. This study used electroencephalography (EEG) to record participants' brain activity changes when experiencing VR sickness. Participants are given VR impressions via screens and head-mounted displays (HMD). In addition, the subject also filled out a simulator sickness questionnaire (SSQ) before and after being given a VR. Brain waves in the alpha frequency range (8 Hz-13 Hz) and low beta frequency (13 Hz-21 Hz) were analyzed through the power spectral density (PSD). From the SSQ results, participants who saw VR through the screen experienced increased nausea symptoms. On the other hand, participants who saw VR through HMD experienced an increase in nausea and oculomotor symptoms (p<0.05). Based on power spectral analysis, changes in alpha wave PSD were obtained in the frontal, central, and parietal brain regions. There was also a shift in the value of the alpha peak frequency from before and after the participants were given VR. The average value of the alpha peak frequency shifts to a significant value. Therefore, it is concluded that the VR sickness from HMD viewing is more significant than through screens in the form of PSD results.

Keywords


Alpha frequency; electroencephalography; low beta frequency, power spectral density; sickness; virtual reality

Full Text:

PDF

References


Y. Cheng, Y. Wang, and W. Zhao, "Shared Virtual Reality Experiences during the COVID-19 Pandemic: Exploring the Gratifications and Effects of Engagement with Immersive Videos," Int J Environ Res Public Health, vol. 19, no. 9, p. 5056, Apr. 2022, doi: 10.3390/ijerph19095056.

E. M. Gegung, “International Tourism and The COVID-19 Pandemic: The Use of Virtual Reality to Increase Tourism Destination Sustainability and How Users Perceive The Authenticity of VR Experiences,†Jurnal Kepariwisataan Indonesia: Jurnal Penelitian dan Pengembangan Kepariwisataan Indonesia, vol. 15, no. 1, pp. 9–15, Jul. 2021, doi: 10.47608/jki.v15i12021.9-15.

A. Mehrfard et al., "Virtual reality technologies for clinical education: evaluation metrics and comparative analysis," Comput Methods Biomech Biomed Eng Imaging Vis, vol. 9, no. 3, pp. 233–242, May 2021, doi: 10.1080/21681163.2020.1835559.

J. Radianti, T. A. Majchrzak, J. Fromm, and I. Wohlgenannt, "A systematic review of immersive virtual reality applications for higher education: Design elements, lessons learned, and research agenda," Comput Educ, vol. 147, p. 103778, Apr. 2020, doi: 10.1016/j.compedu.2019.103778.

E. Chang, H. T. Kim, and B. Yoo, "Virtual Reality Sickness: A Review of Causes and Measurements," Int J Hum Comput Interact, vol. 36, no. 17, pp. 1658–1682, Oct. 2020, doi: 10.1080/10447318.2020.1778351.

U. Laessoe, S. Abrahamsen, S. Zepernick, A. Raunsbaek, and C. Stensen, "Motion sickness and cybersickness – Sensory mismatch," Physiol Behav, vol. 258, p. 114015, Jan. 2023, doi: 10.1016/j.physbeh.2022.114015.

P. Caserman, A. Garcia-Agundez, A. Gámez Zerban, and S. Göbel, "Cybersickness in current-generation virtual reality head-mounted displays: systematic review and outlook," Virtual Real, vol. 25, no. 4, pp. 1153–1170, Dec. 2021, doi: 10.1007/s10055-021-00513-6.

J. Guna et al., "Virtual Reality Sickness and Challenges Behind Different Technology and Content Settings," Mobile Networks and Applications, vol. 25, no. 4, pp. 1436–1445, Aug. 2020, doi: 10.1007/s11036-019-01373-w.

R. K. Kundu, A. Rahman, and S. Paul, "A Study on Sensor System Latency in VR Motion Sickness," Journal of Sensor and Actuator Networks, vol. 10, no. 3, p. 53, Aug. 2021, doi: 10.3390/jsan10030053.

P. Caserman, A. Garcia-Agundez, A. Gámez Zerban, and S. Göbel, "Cybersickness in current-generation virtual reality head-mounted displays: systematic review and outlook," Virtual Real, vol. 25, no. 4, pp. 1153–1170, Dec. 2021, doi: 10.1007/s10055-021-00513-6.

M. Lambooij, M. Fortuin, I. Heynderickx, and W. IJsselsteijn, "Visual Discomfort and Visual Fatigue of Stereoscopic Displays: A Review," Journal of Imaging Science and Technology, vol. 53, no. 3, pp. 30201-1-30201–14, May 2009, doi: 10.2352/J.ImagingSci.Technol.2009.53.3.030201.

C.-Y. Chen, H.-C. Lin, P.-J. Wu, C.-H. Chuang, B.-S. Lin, and C.-H. Lin, "Reducing the discomfort in viewing 3D video with a prism device modified eye convergence," Heliyon, vol. 7, no. 4, p. e06877, Apr. 2021, doi: 10.1016/j.heliyon.2021.e06877.

R. Liu, M. Xu, Y. Zhang, E. Peli, and A. D. Hwang, "A Pilot Study on Electroencephalogram-based Evaluation of Visually Induced Motion Sickness," Journal of Imaging Science and Technology, vol. 64, no. 2, pp. 20501-1-20501–10, Mar. 2020, doi: 10.2352/J.ImagingSci.Technol.2020.64.2.020501.

B. Keshavarz and H. Hecht, "Validating an Efficient Method to Quantify Motion Sickness," Human Factors: The Journal of the Human Factors and Ergonomics Society, vol. 53, no. 4, pp. 415–426, Aug. 2011, doi: 10.1177/0018720811403736.

R. S. Kennedy, N. E. Lane, K. S. Berbaum, and M. G. Lilienthal, "Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness," Int J Aviat Psychol, vol. 3, no. 3, pp. 203–220, Jul. 1993, doi: 10.1207/s15327108ijap0303_3.

S. Sharples, S. Cobb, A. Moody, and J. R. Wilson, "Virtual reality induced symptoms and effects (VRISE): Comparison of head-mounted display (HMD), desktop and projection display systems," Displays, vol. 29, no. 2, pp. 58–69, Mar. 2008, doi: 10.1016/j.displa.2007.09.005.

L. Herman et al., "A Comparison of Monoscopic and Stereoscopic 3D Visualizations: Effect on Spatial Planning in Digital Twins," Remote Sens (Basel), vol. 13, no. 15, p. 2976, Jul. 2021, doi: 10.3390/rs13152976.

J. M. Fulvio, M. Ji, and B. Rokers, "Variations in visual sensitivity predict motion sickness in virtual reality," Entertain Comput, vol. 38, p. 100423, May 2021, doi: 10.1016/j.entcom.2021.100423.

R. S. Kennedy, J. Drexler, and R. C. Kennedy, "Research in visually induced motion sickness," Appl Ergon, vol. 41, no. 4, pp. 494–503, Jul. 2010, doi: 10.1016/j.apergo.2009.11.006.

A. D. Hwang and E. Peli, "Instability of the Perceived World While Watching 3D Stereoscopic Imagery: A likely Source of Motion Sickness Symptoms," Iperception, vol. 5, no. 6, pp. 515–535, Oct. 2014, doi: 10.1068/i0647.

Y. Wei, Y. O. Okazaki, R. H. Y. So, W. C. W. Chu, and K. Kitajo, "Motion sickness-susceptible participants exposed to coherent rotating dot patterns show excessive N2 amplitudes and impaired theta-band phase synchronization," Neuroimage, vol. 202, p. 116028, Nov. 2019, doi: 10.1016/j.neuroimage.2019.116028.

P. M. Podsakoff, S. B. MacKenzie, J.-Y. Lee, and N. P. Podsakoff, "Common method biases in behavioral research: A critical review of the literature and recommended remedies.," Journal of Applied Psychology, vol. 88, no. 5, pp. 879–903, 2003, doi: 10.1037/0021-9010.88.5.879.

T. Gruden et al., "Electrogastrography in Autonomous Vehicles—An Objective Method for Assessment of Motion Sickness in Simulated Driving Environments," Sensors, vol. 21, no. 2, p. 550, Jan. 2021, doi: 10.3390/s21020550.

S. Wibirama, H. A. Nugroho, and K. Hamamoto, "Depth gaze and ECG based frequency dynamics during motion sickness in stereoscopic 3D movie," Entertain Comput, vol. 26, pp. 117–127, May 2018, doi: 10.1016/j.entcom.2018.02.003.

O. D. Kothgassner et al., "Habituation of salivary cortisol and cardiovascular reactivity to a repeated real-life and virtual reality Trier Social Stress Test," Physiol Behav, vol. 242, p. 113618, Dec. 2021, doi: 10.1016/j.physbeh.2021.113618.

H. Ma et al., "Exploring the effect of virtual reality relaxation environment on white coat hypertension in blood pressure measurement," J Biomed Inform, vol. 116, p. 103721, Apr. 2021, doi: 10.1016/j.jbi.2021.103721.

U. A. Chattha, U. I. Janjua, F. Anwar, T. M. Madni, M. F. Cheema, and S. I. Janjua, "Motion Sickness in Virtual Reality: An Empirical Evaluation," IEEE Access, vol. 8, pp. 130486–130499, 2020, doi: 10.1109/ACCESS.2020.3007076.

S. A. E. Nooij, P. Pretto, D. Oberfeld, H. Hecht, and H. H. Bülthoff, "Vection is the main contributor to motion sickness induced by visual yaw rotation: Implications for conflict and eye movement theories," PLoS One, vol. 12, no. 4, p. e0175305, Apr. 2017, doi: 10.1371/journal.pone.0175305.

B. Keshavarz, K. Peck, S. Rezaei, and B. Taati, "Detecting and predicting visually induced motion sickness with physiological measures in combination with machine learning techniques," International Journal of Psychophysiology, vol. 176, pp. 14–26, Jun. 2022, doi: 10.1016/j.ijpsycho.2022.03.006.

C.-Y. Liao, S.-K. Tai, R.-C. Chen, and H. Hendry, "Using EEG and Deep Learning to Predict Motion Sickness Under Wearing a Virtual Reality Device," IEEE Access, vol. 8, pp. 126784–126796, 2020, doi: 10.1109/ACCESS.2020.3008165.

E. Krokos and A. Varshney, "Quantifying VR cybersickness using EEG," Virtual Real, vol. 26, no. 1, pp. 77–89, Mar. 2022, doi: 10.1007/s10055-021-00517-2.

E. H. Henry, C. Bougard, C. Bourdin, and L. Bringoux, "Changes in Electroencephalography Activity of Sensory Areas Linked to Car Sickness in Real Driving Conditions," Front Hum Neurosci, vol. 15, Feb. 2022, doi: 10.3389/fnhum.2021.809714.

K. N. de Winkel, T. M. W. Talsma, and R. Happee, "A meta-analysis of simulator sickness as a function of simulator fidelity," Exp Brain Res, vol. 240, no. 12, pp. 3089–3105, Dec. 2022, doi: 10.1007/s00221-022-06485-6.

M. Almallah, Q. Hussain, N. Reinolsmann, and W. K. M. Alhajyaseen, "Driving simulation sickness and the sense of presence: Correlation and contributing factors," Transp Res Part F Traffic Psychol Behav, vol. 78, pp. 180–193, Apr. 2021, doi: 10.1016/j.trf.2021.02.005.

E. Igoshina, F. A. Russo, R. Shewaga, B. Haycock, and B. Keshavarz, "The relationship between simulator sickness and driving performance in a high-fidelity simulator," Transp Res Part F Traffic Psychol Behav, vol. 89, pp. 478–487, Aug. 2022, doi: 10.1016/j.trf.2022.07.015.

A. D. Hwang, H. Deng, Z. Gao, and E. Peli, "Quantifying Visually Induced Motion Sickness (VIMS) During Stereoscopic 3D Viewing Using Temporal VIMS Rating," Journal of Imaging Science and Technology, vol. 61, no. 6, pp. 60405-1-60405–9, Nov. 2017, doi: 10.2352/J.ImagingSci.Technol.2017.61.6.060405.

M. Abo-Zahhad, S. M. Ahmed, and S. N. Abbas, "A New EEG Acquisition Protocol for Biometric Identification Using Eye Blinking Signals," International Journal of Intelligent Systems and Applications, vol. 7, no. 6, pp. 48–54, May 2015, doi: 10.5815/ijisa.2015.06.05.

R. Srinivasan and P. L. Nunez, "Electroencephalography," in Encyclopedia of Human Behavior, Elsevier, 2012, pp. 15–23. doi: 10.1016/B978-0-12-375000-6.00395-5.

M. Nürnberger, C. Klingner, O. W. Witte, and S. Brodoehl, "Mismatch of Visual-Vestibular Information in Virtual Reality: Is Motion Sickness Part of the Brains Attempt to Reduce the Prediction Error?," Front Hum Neurosci, vol. 15, Oct. 2021, doi: 10.3389/fnhum.2021.757735.

A. Mierau, W. Klimesch, and J. Lefebvre, "State-dependent alpha peak frequency shifts: Experimental evidence, potential mechanisms and functional implications," Neuroscience, vol. 360, pp. 146–154, Sep. 2017, doi: 10.1016/j.neuroscience.2017.07.037.

M. Recenti et al., "Toward Predicting Motion Sickness Using Virtual Reality and a Moving Platform Assessing Brain, Muscles, and Heart Signals," Front Bioeng Biotechnol, vol. 9, Apr. 2021, doi: 10.3389/fbioe.2021.635661.

S. S. Yeo, J. W. Kwon, and S. Y. Park, "EEG-based analysis of various sensory stimulation effects to reduce visually induced motion sickness in virtual reality," Sci Rep, vol. 12, no. 1, p. 18043, Oct. 2022, doi: 10.1038/s41598-022-21307-z.

E. Ugur, B. O. Konukseven, M. Topdag, M. E. Cakmakci, and D. O. Topdag, "Expansion to the Motion Sickness Susceptibility Questionnaire-Short Form: A Cross-Sectional Study," J Audiol Otol, vol. 26, no. 2, pp. 76–82, Apr. 2022, doi: 10.7874/jao.2021.00577.

G. R. F. Suwandi, S. N. Khotimah, F. Haryanto, and S. Suprijadi, "Study of The Effect of Magnetic Fields on Electroencephalography Measurement in Faraday's Cage," Spektra: Jurnal Fisika dan Aplikasinya, vol. 6, no. 2, pp. 101–106, Oct. 2021, doi: 10.21009/SPEKTRA.062.02.

G. R. F. Suwandi, S. N. Khotimah, and Suprijadi, "Electroencephalography Signal Power Spectral Density from Measurements in Room with and Without Faraday Cage: A Comparative Study," J Phys Conf Ser, vol. 2243, no. 1, p. 012002, Jun. 2022, doi: 10.1088/1742-6596/2243/1/012002.

H. Walter, R. Li, J. Munafo, C. Curry, N. Peterson, and N. Peterson, "APAL Coupling Study 2019," Data Repository for the University of Minnesota. 2019.

R. Ramos, J. Arturo Olvera, and I. Olmos, "Analysis of EEG Signal Processing Techniques based on Spectrograms," Research in Computing Science, vol. 145, no. 1, pp. 151–162, Dec. 2017, doi: 10.13053/rcs-145-1-12.

R. Alam, H. Zhao, A. Goodwin, O. Kavehei, and A. McEwan, "Differences in Power Spectral Densities and Phase Quantities Due to Processing of EEG Signals," Sensors, vol. 20, no. 21, p. 6285, Nov. 2020, doi: 10.3390/s20216285.

M. J. Hasan, D. Shon, K. Im, H.-K. Choi, D.-S. Yoo, and J.-M. Kim, "Sleep State Classification Using Power Spectral Density and Residual Neural Network with Multichannel EEG Signals," Applied Sciences, vol. 10, no. 21, p. 7639, Oct. 2020, doi: 10.3390/app10217639.

M. Benda and I. Volosyak, "Peak Detection with Online Electroencephalography (EEG) Artifact Removal for Brain–Computer Interface (BCI) Purposes," Brain Sci, vol. 9, no. 12, p. 347, Nov. 2019, doi: 10.3390/brainsci9120347.

A. J. Furman et al., “Sensorimotor Peak Alpha Frequency Is a Reliable Biomarker of Prolonged Pain Sensitivity,†Cerebral Cortex, vol. 30, no. 12, pp. 6069–6082, Nov. 2020, doi: 10.1093/cercor/bhaa124.

R. S. Kennedy, N. E. Lane, K. S. Berbaum, and M. G. Lilienthal, "Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness," Int J Aviat Psychol, vol. 3, no. 3, pp. 203–220, Jul. 1993, doi: 10.1207/s15327108ijap0303_3.

K. M. Stanney, R. S. Kennedy, and J. M. Drexler, "Cybersickness is Not Simulator Sickness," Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 41, no. 2, pp. 1138–1142, Oct. 1997, doi: 10.1177/107118139704100292.

S. A. A. Naqvi, N. Badruddin, M. A. Jatoi, A. S. Malik, W. Hazabbah, and B. Abdullah, "EEG based time and frequency dynamics analysis of visually induced motion sickness (VIMS)," Australas Phys Eng Sci Med, vol. 38, no. 4, pp. 721–729, Dec. 2015, doi: 10.1007/s13246-015-0379-9.

W. Klimesch, H. Schimke, and G. Pfurtscheller, "Alpha frequency, cognitive load and memory performance," Brain Topogr, vol. 5, no. 3, pp. 241–251, Mar. 1993, doi: 10.1007/BF01128991.

N. Kanayama, M. Hara, and K. Kimura, "Virtual reality alters cortical oscillations related to visuo-tactile integration during rubber hand illusion," Sci Rep, vol. 11, no. 1, p. 1436, Jan. 2021, doi: 10.1038/s41598-020-80807-y.

J. Weber, T. Klein, and V. Abeln, "Shifts in broadband power and alpha peak frequency observed during long-term isolation," Sci Rep, vol. 10, no. 1, p. 17987, Oct. 2020, doi: 10.1038/s41598-020-75127-0.

L. Ma, P. J. Marshall, and W. G. Wright, "The impact of external and internal focus of attention on visual dependence and EEG alpha oscillations during postural control," J Neuroeng Rehabil, vol. 19, no. 1, p. 81, Dec. 2022, doi: 10.1186/s12984-022-01059-7.




DOI: http://dx.doi.org/10.18517/ijaseit.13.4.18165

Refbacks

  • There are currently no refbacks.


Published by INSIGHT - Indonesian Society for Knowledge and Human Development