Systematic Literature Review of Machine Learning in Virtual Reality and Augmented Reality

I Gede Partha Sindu; Rukmi Sari  Hartati; Made  Sudarma; Nyoman  Gunantara

doi:10.23887/janapati.v12i1.60126

Authors

I Gede Partha Sindu Informatics Engineering, Universitas Pendidikan Ganesha https://orcid.org/0000-0002-5470-9942
Rukmi Sari Hartati Faculty of Engineering Universitas Udayana
Made Sudarma Faculty of Engineering Universitas Udayana
Nyoman Gunantara Faculty of Engineering Universitas Udayana

DOI:

https://doi.org/10.23887/janapati.v12i1.60126

Keywords:

machine learning, virtual reality, augmented reality

Abstract

This study aims to conduct a systematic literature review on machine learning methods used in virtual reality (VR) and augmented reality (AR). The method proposed in the SLR consists of three main phases: planning, conducting, and reporting. Researchers used data obtained through Google searches sourced from IEEE Xplore, Springer, Science Direct, Scopus, and Web of Science. The application of inclusion and exclusion criteria is used to select documents. The findings from this study consist of countries involved in machine learning research in the VR and AR fields, machine learning methods, technology used, and work sectors that use machine learning in the VR and AR fields. The results also show that when the machine learning method is used in VR and AR applications, the main advantages are high efficiency and algorithm precision. Moreover, this research observes that machine learning is the most widely applied artificial intelligence scientific technique in VR and AR applications. The results of this systematic literature review show that the combination of machine learning and VR and AR methods contributes to trends, opportunities, and applications in the fields of engineering, arts, education, industry, medicine, tourism, and technology

References

R. Lamb, K. Neumann, and K. A. Linder, “Computers and Education : Artificial Intelligence Real-time prediction of science student learning outcomes using machine learning classification of hemodynamics during virtual reality and online learning sessions,” Comput. Educ. Artif. Intell., vol. 3, no. February, p. 100078, 2022, doi: 10.1016/j.caeai.2022.100078.

P. Lindner, “Better, Virtually: the Past, Present, and Future of Virtual Reality Cognitive Behavior Therapy,” International Journal of Cognitive Therapy, vol. 14, no. 1. 2021, doi: 10.1007/s41811-020-00090-7.

R. Doerner and R. Horst, “Overcoming challenges when teaching hands-on courses about Virtual Reality and Augmented Reality: Methods, techniques and best practice,” Graph. Vis. Comput., vol. 6, p. 200037, 2022, doi: 10.1016/j.gvc.2021.200037.

H. Bai, S. Li, and R. F. Shepherd, “Elastomeric Haptic Devices for Virtual and Augmented Reality,” Advanced Functional Materials, vol. 31, no. 39. 2021, doi: 10.1002/adfm.202009364.

K. M. Chuah, C. J. Chen, and C. S. Teh, “Designing a desktop virtual reality-based learning environment with emotional consideration,” 2010.

M. A. Raya, J. Marín-Morales, M. E. Minissi, G. T. Garcia, L. Abad, and I. A. C. Giglioli, “Machine learning and virtual reality on body movements’ behaviors to classify children with autism spectrum disorder,” J. Clin. Med., vol. 9, no. 5, pp. 1–20, 2020, doi: 10.3390/jcm9051260.

M. Alcañiz Raya et al., “Application of Supervised Machine Learning for Behavioral Biomarkers of Autism Spectrum Disorder Based on Electrodermal Activity and Virtual Reality,” Front. Hum. Neurosci., vol. 14, no. April, 2020, doi: 10.3389/fnhum.2020.00090.

B. Kitchenham and S. M. Charters, “Guidelines for performing Systematic Literature Reviews in Software Engineering Guidelines for performing Systematic Literature Reviews in Software Engineering EBSE Technical Report EBSE-2007-01 Software Engineering Group School of Computer Science and Ma,” no. October 2021, 2007.

A.-W. Harzing, “Publish or Perish,” 2016. https://harzing.com/.

C. Wohlin, “Guidelines for snowballing in systematic literature studies and a replication in software engineering,” ACM Int. Conf. Proceeding Ser., 2014, doi: 10.1145/2601248.2601268.

Z. Lv, D. Chen, R. Lou, and H. Song, “Industrial Security Solution for Virtual Reality,” IEEE Internet Things J., vol. 8, no. 8, pp. 6273–6281, 2021, doi: 10.1109/JIOT.2020.3004469.

J. Asbee, K. Kelly, T. McMahan, and T. D. Parsons, “Machine learning classification analysis for an adaptive virtual reality Stroop task,” Virtual Real., no. January, 2023, doi: 10.1007/s10055-022-00744-1.

T. Palazzolo et al., “Deepdive: Using AI, Machine Learning, and Virtual Reality to Explore Ancient Submerged Civilizations,” Proc. - 2020 3rd Int. Conf. Artif. Intell. Ind. AI4I 2020, pp. 79–82, 2020, doi: 10.1109/AI4I49448.2020.00025.

C. Yildirim, “A Review of Deep Learning Approaches to EEG-Based Classification of Cybersickness in Virtual Reality,” Proc. - 2020 IEEE Int. Conf. Artif. Intell. Virtual Reality, AIVR 2020, pp. 351–357, 2020, doi: 10.1109/AIVR50618.2020.00072.

R. Miotto, F. Wang, S. Wang, X. Jiang, and J. T. Dudley, “Deep learning for healthcare: Review, opportunities and challenges,” Brief. Bioinform., vol. 19, no. 6, pp. 1236–1246, 2017, doi: 10.1093/bib/bbx044.

S. K. Bakshi, S. R. Lin, D. S. W. Ting, M. F. Chiang, and J. Chodosh, “The era of artificial intelligence and virtual reality: Transforming surgical education in ophthalmology,” Br. J. Ophthalmol., vol. 105, no. 10, pp. 1325–1328, 2021, doi: 10.1136/bjophthalmol-2020-316845.

S. Y. Liaw et al., “Artificial intelligence in virtual reality simulation for interprofessional communication training: Mixed method study,” Nurse Educ. Today, vol. 122, no. October 2022, p. 105718, 2023, doi: 10.1016/j.nedt.2023.105718.

F. Zhang, T. Y. Wu, J. S. Pan, G. Ding, and Z. Li, “Human motion recognition based on SVM in VR art media interaction environment,” Human-centric Comput. Inf. Sci., 2019, doi: 10.1186/s13673-019-0203-8.

F. Fathy, Y. Mansour, H. Sabry, M. Refat, and A. Wagdy, “Virtual reality and machine learning for predicting visual attention in a daylit exhibition space : A proof of concept,” Ain Shams Eng. J., no. xxxx, p. 102098, 2022, doi: 10.1016/j.asej.2022.102098.

G. Gomes, “An Emotional Virtual Character : A Deep Learning Approach with Reinforcement Learning,” 2019 21st Symp. Virtual Augment. Real., pp. 223–231, 2019, doi: 10.1109/SVR.2019.00047.

H. Le and M. Nguyen, “applied sciences Augmented Reality and Machine Learning Incorporation Using,” pp. 1–19, 2021.

X. Huang and L. Du, “Fire detection and recognition optimization based on virtual reality video image,” IEEE Access, vol. 8, pp. 77951–77961, 2020, doi: 10.1109/ACCESS.2020.2990224.

G. K. Upadhyay, D. Aggarwal, A. Bansal, and G. Bhola, “Augmented Reality and Machine Learning based Product Identification in Retail using Vuforia and MobileNets,” Proc. 5th Int. Conf. Inven. Comput. Technol. ICICT 2020, pp. 479–485, 2020, doi: 10.1109/ICICT48043.2020.9112490.

K. Sriporn and C. Tsai, “PREDICTING TOURISTS ’ BEHAVIOR OF VIRTUAL MUSEUM USING SUPPORT VECTOR MACHINE WITH FEATURE SELECTION TECHNIQUE,” pp. 1–6, 2020.

V. H. E. W. Brouwer et al., “Heliyon Applying machine learning to dissociate between stroke patients and healthy controls using eye movement features obtained from a virtual reality task,” Heliyon, vol. 8, no. March 2021, p. e09207, 2022, doi: 10.1016/j.heliyon.2022.e09207.

E. Soares, D. Lima, B. M. C. Silva, and G. Teixeira, “Adaptive virtual reality horror games based on Machine learning and player modeling,” Entertain. Comput., vol. 43, no. July, p. 100515, 2022, doi: 10.1016/j.entcom.2022.100515.

L. Wu and L. Chen, “Application of SVM-KNN Network Detection and Virtual Reality in the Visual Design of Artistic Images,” Mob. Inf. Syst., vol. 2022, 2022, doi: 10.1155/2022/7218277.

L. J. Zheng, J. Mountstephens, and J. Teo, “Four-class emotion classification in virtual reality using pupillometry,” J. Big Data, vol. 7, no. 1, 2020, doi: 10.1186/s40537-020-00322-9.

J. Xu, “Immersive Display Design Based on Deep Learning Intelligent VR Technology,” vol. 2022, 2022.

Q. Zheng and Q. Ding, “Exploration of consumer preference based on deep learning neural network model in the immersive marketing environment,” PLoS One, vol. 17, no. 5 May, pp. 1–14, 2022, doi: 10.1371/journal.pone.0268007.

G. Cristina, D. Marco, and G. Xueni, “Immersive machine learning for social attitude detection in virtual reality narrative games,” Virtual Real., vol. 26, no. 4, pp. 1519–1538, 2022, doi: 10.1007/s10055-022-00644-4.

S. M. Hofmann, F. Klotzsche, A. Mariola, V. V. Nikulin, A. Villringer, and M. Gaebler, “Decoding subjective emotional arousal during a naturalistic VR experience from EEG using LSTMs,” Proc. - 2018 IEEE Int. Conf. Artif. Intell. Virtual Reality, AIVR 2018, pp. 128–131, 2019, doi: 10.1109/AIVR.2018.00026.

S. Ishaque, A. Rueda, B. Nguyen, N. Khan, and S. Krishnan, “Physiological Signal Analysis and Classification of Stress from Virtual Reality Video Game,” pp. 0–3, 2020.