HoloLearn: An Interactive Educational System

The HoloLearn project is a sophisticated interactive educational system that attempts to simplify the educational process in the field of medicine, mainly, through the use of Hologram technology. The Hologram technology has been used in conjunction with the feature of user interaction, to take the whole educational process to a completely new level providing students with a different learning experience. The system is more dedicated to medical students as they must study diverse, complicated structures of the human body anatomy and its internal organs. HoloLearn is aimed at replacing the traditional educational techniques with a new one that involves user interaction with real-sized 3D objects. Based on the interview conducted with medical students from different universities and educational levels in Saudi Arabia, and the questionnaire results, it has been found that traditional learning techniques are insufficient/inefficient as they lack quality and most of the criteria that could qualify them to be highly effective as reliable learning materials. Therefore, there is an increasing need for new learning strategies/methods with enough capabilities to give the students the chance to perceive every concept they study, rather than depending on their imagination to picture what a human body looks like from inside, they need visual learning methods. From another perspective, teachers also face difficulties when explaining medical concepts, especially those related to human body structure and behavior. The currently available materials and sources are mostly theoretical. They promote indoctrination and a result-driven approach instead of engaging the students in a process of sharing knowledge and ideas from both parties i.e., teacher and students. In fact, students have to listen and read instead of practicing and exploring, consequently students are prone to loss of concentration and mental distraction during lessons repeatedly, while from another angle they suffer from long study hours and difficulties in retrieving information. The results of this project indicate that when combining the Hologram technology with the user interaction feature, the educational process can be highly improved, and can be much more creative and entertaining. Keywords—Interactive educational system; hologram technology; user interaction


I. INTRODUCTION
The world is changing, and technology is contributing to almost everything. Starting with the massive growth of the Internet that connected people from all over the world, the use of mobile applications to simplify daily tasks, and ending with the use of Information Technology in almost every field, industrial, medical, agricultural, and educational. The need to facilitate the process of education is continuously growing and interactive learning methodologies are becoming an urgent requirement. Every day, people try to find new ways to convey/deliver information in the simplest possible form to enhance long-term memorization while maintaining the information correctness, and here comes the importance of this project. For instance, medical students face daily problems trying to find efficient learning methods and reliable sources of information that simplify the content and provide it in an understandable form. Text-based information, images, videos, models, or even Cadavers, which, according to [1], "remain a principal teaching tool for anatomists and medical educators teaching gross anatomy", are also known as corpses. All these different methods are for the sake of a better understanding of human body structures and different health conditions. Each one of those methods has its own limitations that prevent it from providing enough information, students are still in a continuous search for alternatives. Specifically, medical students from different levels face problems of distinguishing between different body organs and its functions, understanding muscles" movement, body behavior, and how it responds to external stimuli. Memorizing all of these details and applying this knowledge when needed would be a very complicated process if not supported by effective learning methods. Some universities try to simplify the process by using models or corpses. One of the drawbacks of using a corpse is its incapability to demonstrate vital activities. For instance, a corpse will not be able to show blood circulation, heartbeats, or neural flow. On a global scale, it is agreed that the use of corpses in education is not the ideal option. As stated in [2], "The principal disadvantages of the anatomy room embalming solution are the brown discoloration caused by phenol and the irritation to the dissector's eyes and lungs by formalin. Each anatomy department has its own preference for the preparation, concentration of fluids, and the manner in which they are injected. The 'standard' method of preservation outlined is ideal in that bodies can be kept for months at room temperature but on the other hand, it alters the normal color and consistency of the tissues, rendering them less life-like. Most flap dissection workshops last between 2 and 4 days so fresh cadavers are suitable and seem preferable. One should not however forget that fresh cadavers are not sterilized. They must be very carefully selected so that bodies contaminated with HIV, hepatitis, and prions are excluded. Prions or the socal". HoloLearn proposes a different method of education that 432 | P a g e www.ijacsa.thesai.org does not require the use of real corpses, which is learning by interacting with real-sized 3D objects that simulate the human body organs accurately. Our proposed solution aims at facilitating the educational process with the use of the Hologram technology, which is according to [3] is "a photographic method that records the light dispersed from a body and then produces a realistic image identified as threedimensional Hologram", combined with user interaction. Using the Hologram technology will give the opportunity to display each organ as a 3D object, show its movement, and allow interaction with it, such as selection, rotation, and zooming. Indeed, this method will provide the students with a better and visually appealing experience, ensure full understanding, and enhance long-term memorization. The aim of this project is supported by the fact that auditorial memory is inferior to visual memory according to most research.
HoloLearn project can serve in many fields, including universities and schools to be used by both, teachers, and students as an educational tool where they can see how the actual human body looks from inside and visualize how it behaves under different conditions. Moreover, it can be utilized in hospitals to be used by physicians in surgeries to specify the parts/organs they should work on precisely. Also, can be used by physicians to provide the patient with better visualization of the disease diagnosis.
HoloLearn project is intended to develop an environment with an educational purpose in which users, specifically medical students, can interact with real-sized 3D objects that simulate human body organs. The system consists of three main parts. The first part includes displaying the complex structures of the human body as a Holographic model, including the main systems, such as Muscular, Skeletal, Venous, and Skin. Moreover, it includes the internal organs, such as the heart, lungs, and kidneys. The second part is concerned with the main feature we are interested in, which is the user interaction. This part consists of allowing and capturing some pre-defined movements/gestures made by the user against the model through a specialized sensor (Kinect V2), where a desktop application will be used to process and analyze the received interaction. The third part guarantees that the system reacts in accordance with the gesture made by the user. The reaction is mainly a change in the Hologram model position and appearance to better achieve the interactivity and provide the students with a different visual learning experience.
The project is aimed at contributing to the improvement of the educational process and taking it to a completely different level by replacing traditional educational techniques with a new one that ensures user interaction with real-sized 3D objects, presented using the Hologram technology, where multiple students can participate in the learning process simultaneously and with the least possible number of devices and wear. This paper is organized as follows: In Section II, an overview of the problem domain and the related work is discussed and followed by the actual implementation of the purposed solution, regarding the hardware and software, in Section III. The obtained results, achieved objectives, and the shortcoming and challenges are argued in Sections IV and V. finally, the future work is discussed in Section VI.

II. BACKGROUND
The educational methods have evolved rapidly in the past 10 years. In the field of medicine, the students used to depend on paper resources, which may include images to help visualize the structures. These resources could not give the students the ability to understand every concept or visualize the inner structure of the human body comprehensively. The way the human body behaves from inside, the blood flow, the nerve supply, and the way each organ moves, all such important details could not be described sufficiently using books, images, or cadavers. The students were partially, and in some cases totally, incapable of understanding such important details. Later on, the technology started to intervene. 3D videos, animated pictures, and some mobile applications have been developed to serve the purpose of facilitating the educational process. Each one of these methods has its own limitations that increased the need for a new technique. For instance, most of the current mobile applications that are dedicated to medical students either lack interaction or provide objects that can be only displayed in a very small size (usually on mobile screens). Moreover, most of them require a monthly or annual paid subscription. On the other hand, they are not provided by the university as an official source of education or used by teachers in classes, rather, the students themselves are responsible for searching and finding such applications in order to follow up with the lectures and survive the course, as the traditional techniques are no longer useful enough. Now-a-days, with the simultaneous advancement of technology and education at a time, it has become possible to build a learning environment that gives the students the chance to be more involved. Instead of watching, listening, and receiving information in a unidirectional process, they finally can intervene more in the education, turning the whole process into a bidirectional experience. In the past few years, many technologies have been used in order to provide a highquality education, such as Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR) technologies.
One of the organizations developed a solution, HoloHuman [4], based on the use of Microsoft HoloLens, where the user has to wear such a lens to become involved in a virtual environment that is built upon Mixed Reality technology. This solution allows users to interact directly with a real-sized 3D model that simulates the actual human body, once the lens is on. Another distinguishing feature of this solution is the ability to accommodate multiple users at once, with the condition of wearing the lens to get involved in the virtual environment. Another interested organization developed a mobile application, called Essential Anatomy 5 [5], to display human body anatomy including the male and female body variations as separated 3D models on screen, where users can carry all the functions at their fingertips. This application contains 11 complete systems that can be accessed from the main screen. "These include Muscular, Skeletal, Connective Tissue, Venous, Arterial (Including the Heart), Nervous (Including the Brain), Respiratory, Digestive, Lymphatic, Urogenital, and Skin" [5]. To support students, www.ijacsa.thesai.org they created a Test-Your-Memory feature, in which a user can use multiple options to evaluate his knowledge. This application works on mobiles with no need for any extra devices, yet it lacks full interactivity and user involvement. Anatomy ARVR is another mobile application with an educational purpose [6]. As the name implies, it uses virtual reality and augmented reality to offer students an interactive learning environment. In this application, the user must position the camera in a certain manner on the ARVR book provided by the same company to visualize the human body organs as 3D objects. This application provides the students with the ability to rotate, zoom and view details, such as the name and function of a selected organ to achieve a better experience. Comparatively, BioDigital, according to [7], is a "software that power's the world understanding of human body anatomy through interactive 3D software platform" for visualizing anatomy, disease, and treatment. In addition to the usual features, such as visualizing human body anatomy in 3D, and allowed interaction, this application has covered the different health conditions, disease diagnoses, and treatment in its scope. The BoneBox™ Skull Viewer is another example of the use of technology to facilitate the educational process in the field of medicine, however, it has a limited scope where it only focuses on the skull anatomy and provides a labeled, detailed description of the type of functions and processes that run inside [8].
The problem of this paper has been addressed multiple times already. However, each of the existing solutions has some limitations that we are trying to overcome in our proposed solution. The need for each user to wear/hold extra devices to finally get involved in the virtual environment has been eliminated by the use of one specialized sensor, Kinect V2, and one projector to display the final Holographic model, which is unlike AR and VR, can be seen by everyone without 3D glasses. This solution makes it possible to place the whole system in one spot at laboratories in schools and universities to be used as an official educational tool. Virtual Reality, Augmented Realty and Mixed Reality technologies have been replaced by Hologram technology to provide a more realistic viewing experience through a large-sized 3D Holographic model. To achieve better interactivity and allow students to take charge of the learning process, reduce the need for an instructor to be present, and promote self-learning, the Kinect V2 sensor has been used. This device will capture and detect any pre-defined gestures made by the user from a reasonable distance, where students can keep in a continuous process of learning through sending and receiving reactions without any manual intervention from a facilitator. From another point of view, the scope of this solution covers the entire human body.
HoloLearn is intended to make a revolution in the field of education. Whether it is the medical, biological, geographical, or engineering school this project will encourage institutes and universities to start allowing technology to intervene more in the evolution of the educational process. Specifically, they will start benefiting from the Hologram technology as it has so many applications and high potential to emulate almost anything.

III. IMPLEMENTATION AND RESULTS
The following subsections discuss the actual implementation of the proposed solution and the individual tools utilized to produce the final working system, including both hardware and software.

A. Hardware Set-up and Installation
In the context of Holographic models, different tools and devices can be suggested to produce a Holographic-like image i.e., Holographic 3D model. The author in [9] defines Holographic displays as "a device that uses light diffraction to create three-dimensional (3D) images in space. When real objects and holographic images are located in the same space, they can be perceived without inhomogeneity". Devices range from low-quality ones, that are capable of producing exclusively small-sized 3D models, that lack flexibility and cannot be extended to incorporate other beneficial features such as user interaction, to high-quality devices that are extremely expensive. Most of the currently available devices are considered to be one-way displays that are incapable of receiving and processing any type of interaction. They do not include any built-in sensors or detectors to recognize human movement. Moreover, they do not consider future scalability and integration with other devices; therefore, they are not physically prepared for such purposes. For instance, there is not any kind of ports or adapters to support possible integration. The traditional basic method of displaying a Holographic model consists of using a pyramid-shaped stage that comes in various sizes in which there is a need for a specific type of tablet to store the 3D objects to be displayed as Holographic images. Other types of Hologram projection devices are "composed of a fan that produces an illusion of 3D objects floating in the air. The fast-spinning fan becomes nearly invisible to the naked eye and the projected object thus gets a see-through background" [10]. Most of the time, these devices are used for advertising and marketing purposes to showcase products in shopping malls and supermarkets. Obviously, the main drawback of the discussed methods is the very small size of the produced models and the questionable quality, thus the incapability to reflect reality precisely or allow users to perceive fairy images without eyestrain. The size of the produced model is one of the most concerned criteria when it comes to selecting a device to display the Holographic model. On the other hand, some of the devices come with a pre-developed mobile application to display only the objects that can be consumed and processed by that application, which prevents any further utilization of such devices. In some rare cases, other devices ship with flash memory that, again, contains all the 3D objects to be displayed as Holographic models.
Hence, none of the above-explained devices fit with the HoloLearn's main purpose, which is being able to output realsized 3D models while making it possible to receive and analyze user interaction. The main problem of this paper is not focused on displaying Holographic models as this issue has been addressed and solved years ago. On the contrary, it is focused on utilizing this technology and benefiting from the existing studies and tools, i.e., Hologram projectors, to develop a visually appealing environment that adapts the concept of user-engagement in a visual and real learning www.ijacsa.thesai.org experience to help a better understanding and encourage a different method of education.
The reasons and facts showed earlier prove the inadequacy of the existing devices and the increasing necessity of a new customized device that fulfills the new requirements. Our customized display stage is inspired by the related work and previously developed Hologram projectors. It considers their drawbacks and tries to come up with a holistic solution that covers all the aspects and defects.
A Holographic model can be produced through Holographic 3D videos. A 3D video is essentially composed of a 3D object reflected four times. The reflections are made with respect to very precise values of different attributes, such as distance, angle, and tilt to end up with a model displayed correctly in the center. Reflection is a key to produce Holographic models.
Creating a 3D object picturizing each organ, and then reflecting each created object four times on four different sides manually, though various software applications that can accomplish such a task automatically do exist, but the quality is uncertain, to produce a Holographic video would be a long time-consuming process. From another perspective, these videos usually require to be pre-built and ready to be displayed. They cannot be created/delivered on demand. Hence, it would be more complicated to allow interaction with a pre-created set of videos where each video should be displayed according to the gesture/movement done by the user to achieve interactivity, and where there is no time-ordered execution. Especially in those devices where the videos are stored in a flash memory that does not contain any programmed software capable of analyzing the received interaction and selecting the appropriate Holographic video amongst the existing ones to provide a reaction back, i.e., another view of the same Holographic model.
To overcome these obstacles creating a Hologram projection without sacrificing the realistic simulation and the promised quality, we proposed a different solution that uses four replicated projectors, i.e., the usual projectors used in classes to display slideshows, which defined in [11] as "an output device that takes images generated by a computer or Blu-ray player and reproduce them by projection onto a screen, wall, or another surface".
As explained earlier, a Holographic video is composed of a set of reflections of the same 3D object considering very precise measurements. Our proposed solution takes advantage of the pre-designed 3D objects that simulate human body organs provided by the Unity 3D framework, builds the full view of the human body by positioning each organ according to the actual anatomy, and omits the need to reflect each 3D object four times. The exclusion of the reflection step results from the use of four projectors where each one will display a different view of the same 3D object to eventually create the Holographic effect.
The proposed solution is built upon the use of the pyramidshaped stage, which is the primitive method of displaying Holographic models. To bring realist-looking images to life and allow them to flow in space in real-size, the size of the stage has been doubled almost 33 times, the measurements have been made in proportion to the small-size Hologram pyramid shown in Fig. 1. With the help of an expert engineer, the stage has been built from scratch to achieve the objective of this project.
The material, size, height, and thickness of the acrylicmade stage have been selected with extreme care to satisfy the requirements of this project. The decision as to which type of glazing should be used has been made upon the need for antibreaking plates, shatter, and scratch-resistant. Acrylic (PMMA) in comparison to glass is harder to break and can be extremely reflective, which fits the project's purpose of reflecting 3D objects on transparent panels to produce the Holographic model. The degree of slope, the angles between the plates, the angles between the pyramid and the underlying surface are all important factors that have been considered in producing the final Holographic effect.
Indeed, we have omitted the reflection as an editing step during the preparation of the 3D objects before displaying, but the suggested alternative is to use four image projectors, at the production phase, to be positioned at the top corners of the stage. The view angle and the direction of each projector play an important role in outputting each side of the same 3D object correctly on the corresponding plate to form the final model. In other words, the four projectors replace the need for the four digital reflections.
The second part of this system is focused on providing interactivity and encouraging user engagement, which can be achieved through the use of the Kinect V2 sensor that is developed by Microsoft and defined as a "sensor that captures hand and body movements. Kinect can work only with Windows operating systems. Windows Kinect version 2.0 allows developers to create applications that need voice recognition, depth camera or face-tracking to capture the movement of the human body" as explained in [12]. The use of this sensor will allow the system to recognize any gesture/movement made by a human from a reasonable distance. It does not require users to be too close, thus being able to extend the range of detectable gestures and relief the users from the uncomfortableness of being surrounded by devices.
The general architecture of the hardware implementation is shown in Fig. 2.

B. Software Implementation and Construction
Given the main requirements of the system, which focus on a realistic simulation of human body anatomy and user engagement obtained through the use of the Kinect V2 sensor, it has been decided to develop a desktop application. This decision is based on the constraint of Kinect V2 devices that require a certain type of port. It only works with USB 3.0 ports and Windows OS. From another angle, the use of four image projectors also requires a connection with a PC/Laptop. Due to the mentioned restrictions, the use of mobiles in this system would be challenging.
As defined in [13], Unity3D Framework is a "crossplatform environment for developing multidimensional Virtual Reality (VR), and AR video games on mobile, desktop, and web platforms". This framework offers developers a set of pre-built 3D objects, in what they call a Unity Asset Store. These objects simulate numerous realworld entities and can be purchased at affordable prices. In our proposed solution we have taken the advantage of this service to build our own complete human body 3D model that composes different systems such as Skeletal, Muscular, Skin, and Venous. Similarly, at the organs level, different internal organs have been simulated, such as the heart and lungs. The internal organs have been placed accurately according to the actual anatomy, taking into consideration depth, distance, and size as our model is not flat.
To accomplish the interactivity, we have made the best use of an open-source pre-developed library, called KinectUnityV.0001. In addition to allowing seamless integration between the sensor and the 3D model, it defines different classes, objects, and functions to facilitate the process of recognizing gestures made by users to be further processed and analyzed. This library gave the chance not only to detect a gesture, e.g., handgrip, but to determine the exact measurements and position of the detected gesture on x, y, and z axes along with other values, such as the angle. This library helped to define detectable gestures and assigning corresponding functions, to be made by users against the model, as follows: 1) Right-hand-open for selecting a specific system/organ. 2) Right-hand grip for dragging a selected organ.
3) Right-hand circulation around the elbow for rotating a selected organ.

4)
Two hands pinch/apart for zooming a selected organ. 5) Right-hand-lasso for switching between functions. These five functions compose the possible interaction that will allow users to perceive organs from all sides and provide them with a holistic view of the human anatomy. Visual Studio Code has been used as the main environment to analyze the received actions and program reactions accordingly with the help of the C# programming language. C# language is the ideal option for the purpose of this paper, as it is one of the dominant languages in the field of visual systems that may involve technologies like Virtual Reality, Augmented Reality, or Holograms.
The system starts by setting up the environment and getting the Kinect V2 sensor, the projectors, and the desktop application ready to operate. The sensor detects most of the actions and gestures made by the user against the model. On the other side, the application on the backend is waiting to receive one of the five defined gestures. It ignores all other gestures/movements that are not defined in the system. Moreover, it ignores defined gestures that are made outside the boundaries of the defined area/zone. The height in this process plays an important role. For instance, if the user releases his right hand at a very low height, he will be ignored and receive no reaction back.
Once a pre-defined gesture, e.g., right-hand release, at a specific height, i.e., inside the defined area, is detected, the corresponding function is executed. To achieve interactivity, the Hologram model view changes accordingly. Based on the type of gesture and the measurements, such as the position, angle, and height of the hand, the object might be selected, dragged, rotated, or zoomed. The workflow of each function is explained separately in the following paragraphs.
The user begins with selecting a specific body part, e.g., Muscular system or Heart by releasing the right hand and moving it forward to the model; accordingly, each time he continues to move his hand further, the next part will be shown along with the name, while the previous one will be hidden. The body parts are shown sequentially in a predefined and certain order. Likewise, the user can follow the same steps to reselect the preceding body part, except that he will have to move his hand back toward his shoulder.
To avoid confusion and keep the user concentrated, the right hand has been nominated to carry most of the functions. To get a closer look at one of the body parts, the user can hold his hand in a grip position to drag the selected organ to a specific point in the space. Relatively, the user can rotate the selected part around the y-axis by moving his right hand in a circular way around the elbow, where the angles of the object will change accordingly, and the object will appear to rotate and float freely. Moreover, a user can pinch and apart his hands to adjust the zooming and view the details of a selected organ as one of the functions provided in HoloLearn.
One of the most concerning issues in this solution is the transitioning between functions. As many functions can be applied to the same object, e.g., an object can be selected, dragged, and then rotated, there has been a need for a distinguishing gesture to define the abortion and transition www.ijacsa.thesai.org from one function to the other. For this purpose, a newly customized gesture, namely lasso, has been developed and programmed so that users can apply multiple functions without ambiguity. For instance, a user can select an organ and then hold his hand in a lasso position to move to the next function, which is dragging, and start dragging that organ. Similar to organs, the functions are ordered in a certain manner that is selection, dragging, rotation and zooming.

IV. DISCUSSION
HoloLearn is an educational system that facilitates delivering medical concepts using Hologram technology incorporated with allowed user interaction. The main targeted users are medical students as they face many problems trying to understand complicated concepts including the different human body structures. The aim of the system is to be able to replace traditional educational techniques with a new one that utilizes technology while adapts visual and interactive methods of learning, which makes a revolution in the field of education and encourage others to innovate new educational methods based on the use of the Hologram technology as it has the potential to make a significant positive impact.
In earlier studies, it was possible to either interact with small-sized 3D objects that are incapable of providing a realistic experience, while wearing and holding many devices or view 3D objects with no interaction at all. Through the architecture proposed in this paper, such problems are solved. Our architecture combines human body simulation and focuses on allowing real-sized simulation as it has been a compromised feature in earlier solutions, with user engagement in one integral system. The implementation of this system has proven to be succeeded to serve the aim intended for the project, which is to engage students in a visual interactive environment that helps students to take the lead of the educational process and start educating themselves, reducing the reliance on teachers.

V. CONCLUSION
This system made it possible to turn education from a tedious process of long classes where students have to listen and write down information, into an entertaining one that allows students to take a bigger part by exploring, practicing, and running an endless loop of failing and trying.
From another point of view, this system has eliminated the need for attending anatomical classes at foul-smelling laboratories full of dead bodies that are reserved at a very low temperature to extend their shelf lives as long as possible.

A. Challenges and Limitations
Finding a Hologram display with enough capabilities to output real-sized 3D models was challenging as most devices are meant for commercial purposes. Besides that, most of these devices are intended only for displaying and cannot be extended to include other advanced features, such as user interaction. The scope of the project commits to offer an atmosphere of interactivity; this requirement has been satisfied through the use of an aided device, i.e., Kinect V2 sensor. Such a sensor requires exclusively a USB 3.0 port to be connected. On the other hand, mobiles do not fit the purpose of the system as they cannot satisfy the need to capture user interaction and produce the promised results. These shortcomings have been overcome by building a customized Hologram display that allows the incorporation of supplementary devices and is qualified for displaying largesized models.

VI. FUTURE WORK
Regarding the future work, after implementing all functions included in the scope successfully, the long-term goal is to extend the system by including deeper complex human body structures. Moreover, advanced features can be developed, such as showing the heart pulse, blood circulation, blood supply, nerve supply, and afferent/efferent neurons in order to provide students with a complete guide in the field of medicine. The ultimate goal is to certify the system as an official educational method, where people apply the exact concept that is built upon the use of Hologram technology, in different majors of education.