Improve Speed Real-Time Rendering in Mixed Reality HOLOLENS during Training

Augmented reality (AR), virtual reality (VR), and mixed reality (MR) are advanced applications of computer visualization Which a hybrid structure allows users to explore novel used with other technologies in healthcare among other sectors give a promising future owing to the capabilities that come along with the technology that enables the medical personnel can carry out their surgical operations precisely. HOLOLENS 1, an MR product by Microsoft, is one of the first AR devices that have been widely applied in medicine for the treatment of complex diseases. It is also applied in operations that require a lot of care, for example, surgery of the liver. It is the main objective in the research to use HOLOLENS in performing surgeries while increasing Time Interval and controlling Semantic Segmentation while maintaining the truth of patient liver data during surgery for the segmentation liver 3d model. Next, we describe a new technology that increases the points of light, and the more the 3D intensity, the brighter the images the easier to interact with them. Holographic intensity is also to avoid blurring images to the point where the user sees through transparent HOLOLENS lenses. This improves Time Interval lens sensitivity and user detection in the environment. Finally, we describe a new framework for improving speed realtime render and model segmentation used hybrid Visualization between VR & AR called MR in which we decree render time speed through increase point light throw color calculations and energy function to be fast in sending and receiving data via WIFI


I. INTRODUCTION
Augmented reality (AR), virtual reality (VR) and mixed reality (MR) are the applications of computer visualization at different levels. There has been a mutation of technology starting from virtual reality via augmented reality and then to the recent mixed reality. Virtual reality immerses the user in a virtual world such that the user feels as if he or she is interacting with real-world objects [1]. This, however, is an illusion since the visualization is only made to behave like real objects. The applications of virtual reality can be seen in games like Pokémon go and training in some fields like engineering. Augmented reality took the second step after virtual reality by bringing in some reality, for example, the use of computed tomography (CT) scans that give a 3D transformation of an image as argued by [2]. This means that medical operations can be easily carried out due to the geometrical possibilities and ease of use. It has also extended mentoring and learning by giving efficient practical techniques to novice learners, for example, medics in the field of medicine and health.
MR is a conjunction between the AR and VR such that a real-world object can be maneuvered with the reference of a virtual object. In the medical sector, a patient can be operated in real-time using computer visualization the best being the HOLOLENS. Surgical operations that involve a lot of care, most notably, brain, liver, and other essential body organs can be carried out with a greater level of accuracy. The visualization from the eye of medical personnel is displayed on the screen which necessitates insights from other team members which will enhance accuracy further. Moreover, training of interns or novice personnel is made Easier since they can interact with augmented objects hence the most effective and fulfilled learning. This paper will focus on computer techniques and visualization regarding HOLOLENS and how it can be improved to enhance its Usefulness in liver surgery [3].
II. BACKGROUND RESEARCH HOLOLENS, specifically Microsoft HOLOLENS, is a mixed reality device that comes as a pair of smart glasses. Microsoft HOLOLENS is still under development and hence named Project Baraboo which was first released to the users in Canada and the United States. It runs under the operating system platform of windows 10 and is well known for being the first head-mounted device giving output as display and sound. The display is in the form of 3D images and hence easier to manipulate images as if they were real-world objects. Kinect was an inspiration that was used to come up with the tracking technology of the HOLOLENS which was the Xbox console, a gaming product, also by Microsoft.
Investors, that is, Asus in conjunction with Samsung, have shown interest in HOLOLENS and have pledged to improve the hardware part of the product alongside the concept that was www.ijacsa.thesai.org used to come up with the product. Moreover, Microsoft has made the product accessible to the public through content management systems, for instance, GITHUB and Bit Bucket, to facilitate improvement from developers and programmers all over the world [1].
HOLOLENS was designed such that it can be tilted in an up and down or backward and forward manner by a cushioned headband. It requires to be mounted on the head and gives input in the form of a display which then can be adjusted by the headband which lies from the inside. The HOLOLENS has an adjustment wheel that a user can use to ensure HOLOLENS is comfortable in the head. The adjustment wheel distributes the weight that the whole unit has and also acts as support which acts as a prerequisite to the tilting toward the eyes at the front of the head. In comparison, unlike the backside that contains the adjustment wheel, the front unit consists of sensors, processors, projecting lenses and cameras among other related hardware. Also contained in the visor, which is tinted and consists of a pair of combiner lenses which is transparent acts as an output where images get displayed. The images are not displayed in the whole area of the visor but only the lower half. The calibration of the HOLOLENS is to the nearest IPD, which stands for inter pupillary distance, or it can be accustomed according to the user vision.
According to Joachimczak (2017), a pair of 3D speakers are located close to the ears of the user and distributed along the side edges. Audio speakers are competing with the typical sounds but give no obstruction to the sounds from the external environment hence the user will not hear the mix of virtual and environmental sounds, meaning there is no distraction at all. Binaural audio is generated by the HOLOLENS which is known to aid in the simulation of spatial effects. This means that the user can virtually perceive and estimate the location of a sound as if originating from a location that is from a virtual location or pinpoints [4].
The hardware part of the HOLOLENS consists of inertial measurement unit (IMU) [5] [6] Fig. 1, that is, the collection of a magnetometer, an accelerometer in conjunction with the gyroscope, four intelligent sensors distributed in pairs on every side, a depth camera which is energy-saving with a relatively wide angle of view which is 120 degrees by 120 degrees and photographic camera which can record videos. It also consists of an array of four microphones and light sensors with ambient characteristics.
HOLOLENS comes in handy with the HPU, which is a holographic processing unit that adds the functionality to the Intel Cherry SoC which contains the CPU alongside the GPU. HPU is a core processor that was solely manufactured or tailored to the HOLOLENS hence may not work in other hardware devices that do not have relative characteristics as the HOLOLENS. The HPU, just like the SoC, has LPDDR3 with a size of 1GB. Both HPU and SoC commonly share an SRAM of 8MB and in that case, SoC controls 64GB eMMC alongside operating under the windows 10 environment.
Moreover, the HPU utilizes 28 DSPs customized from Tensilica for processing and integrating the data received by the sensors alongside task handling techniques like recognition of gestures, spatial mapping, speech and voice recognition. A rechargeable battery is contained in the HOLOLENS with a lifespan of approximately 3 hours when actively used and approximately 2 weeks when left in standby mode. This shows that the HOLOLENS may not be used in an environment with no power supply, but the advantage is that once the power supply is adequate, the device can be charged while being used at the same time.
HOLOLENS supports IEEE802.11 ac which is Wi-Fi connectivity, one of the wireless local area networks (WLAN). It also features Bluetooth V4.1wireless connectivity which consumes less energy hence termed LE, low energy. Bluetooth LE is utilized in the headset for pairing with Clicker, which is an operating input device with the size of a thumb. The Clicker is used for scrolling the interface and selection of simulating objects when necessary. The clicker consists of a clickable surface for selection and sensors oriented which provides scrolling functions that may be in the form of panning or tilting the unit [3].
Elastic finger loop and USB micro-B version 2.0 are also contained in the Clicker which functions as device holder and internal battery charging respectively. Display and volume buttons for brightness are located at the top edge. Display buttons which are used to control brightness are near the left ear while volume buttons are on the right ear. There are adjacent buttons which are varying in shapes to make it easier for the user to distinguish when they touch, that is, concave and convex.
All these sums up to the HOLOLENS a Microsoft product that had several applications in the market. The HOLOLENS has its strengths and weaknesses which can be easily overcome with the advancement in technology and hence the reconstruction of the product to fit the required standards. The applications of HOLOLENS can be seen in windows 10 products like CORTANA, and other products like HOLOSTUDIO, HOLOTOUR, ACTIONGRAM, ROBORAID among other products. It has also been applied in various sectors of the economy, for instance, health, education, business and also in technological advancements. Mentoring or training novice medical students has been made easier with the HOLOLENS which gives CT scans that a student can learn at an instant. HOLOLENS, however, has some limitations, for instance, the lower storage, poor battery life, its limitation of usage to only the inside of a room or lab and poor rendering of images. This means that the HOLOLENS requires some adjustments to ensure that it performs as expected and to avoid disastrous effects that may come with the limitations it has. Microsoft team of developers issued the source code of the project to the public in GitHub to aid in the improvement of the project.
The use of Unity and reprogramming of the source code to ensure improved rendering and the readjustment of the device ergonomics will be a success to the project. The release of HOLOLENS is an improved version of HOLOLENS though it still has some limitations like rendering and speed or recognition of gestures in some instances.

A. Augmented Reality (AR), Virtual Reality (VR) and Mixed Reality (MR)
Augmented Reality (AR) provides the user with the direct environment and combines additional data or information in the form of graphics amalgamated with the real objects. The object will then appear as an overlay and the user perception is somehow made to be an illusion by the device hence the user feels as if they are interacting with real-world objects [7]. Virtual Reality (VR), on the other hand, involves the simulation of a real or virtual environment which allows the user to interact unobtrusively that is as if the object is physically present. The only difference with AR is that VR does not alter the way the user views the object hence commonly referred to as a see-through technology as argued by Kowalski.
AR also has a wide variety of applications due to its easy interactivity and the way it brings the object to look as if they are real in conjunction with its cost-effectiveness. AR allows the creation of 3D shapes and can be manipulated virtually by applying geometrical operations like rotation, reflection, sheering among other operations. Evans claims that Mixed Reality (MR) is the technology that utilizes both AR and VR hence making the object to be easily manipulated, for instance, the HOLOLENS used in surgical operations and other engineering operations [8]. When all the three technologies are combined, they give groundbreaking solutions to the health sector in terms of efficient health services, education and the improvement of health operations altogether.

B. Applications of AR, VR, and MR in Medicine and Health
Care AR, VR, and MR have been applied in medicine and health care in several ways which include carrying out medical operations like surgery, medical records and scheduling, mentoring novice medical students and medical predictions among other several applications. Surgical operations are among the day to day activities of medical personnel that require maximum attention and accurate results. Chaotic environments like trauma sections are also difficult to be operated with the use of manual paraphernalia in the case where feedback is needed instantly [9].
The use of MR has facilitated the operations by allowing the professional to make decisions quickly and get insights to form the team from the visualization. HOLOLENS is an example of an MR device that has been widely applied in the health sector. Considering a chaotic environment, MR devices like HOLOLENS are suitable since they can be easily carried from one place to another without being connected to another device, that is, they are wireless [10].

C. Nature of Liver Surgery
The liver is the second largest body organ preceded by the skin. Liver surgery is one of the challenging surgical operations like those of the heart and the brain among other delicate body parts. This is because vital blood vessels that lead back and forth the heart have their way via the liver. Considering the liver itself, it is an organ that is fleshy hence can easily tear followed by serious bleeding when injured. Bloodless surgery is almost impossible with the liver though with the advancement in technology and the professional nature of medical personnel, bloodless bleeding can become almost probable [11].
According to Sharma, liver surgery, however, can only be carried out under some circumstances, for example, early stages of liver cancer or tumors in the liver. Liver transplant is also another instance where in some cases the whole liver is malfunctioned and hence the need to replace it altogether. The main advantage that comes with the liver is that it can regenerate and hence if the cancer cells have not spread enough in the organ, the affected part can be resection and the organ will continue to function normally [1]. Therefore, before liver re-sectioning can be performed, a lot of care should be taken and the high precision of the operation is demanded. This is where mixed reality technologies like HOLOLENS among other biotechnologies come into play.

D. The use of HOLOLENS in Liver Surgery
Due to the delicate nature of liver surgery, HOLOLENS comes in handy as a device that makes liver surgery easier and more precise [12]. Liver tumors or cancerous growths may involve a small area of the liver but require maximum care considering that the liver is highly vascularized and delicate to the point that a small surgical mistake could lead to serious repercussions. HOLOLENS is used to apply mixed reality technology such that a liver that needs operation is operated virtually and projected on a screen [13]. This allows other professionals to give insights at an instant while interacting with 3D objects that mimic the patient's liver. The medical personnel may also be in different geographical locations but can give insights or even carry out operations themselves. With the use of the HOLOLENS, the bloodless surgery can almost be attained but considering the nature of the liver, the technology can only reduce the degree of blood in the surgery and also makes it convenient to apply surgical operations on patients.

E. Strengths of HOLOLENS
HOLOLENS is a mixed reality holographic computer by Microsoft and is far the best device that has a promising future www.ijacsa.thesai.org for many fields including medicine [13]. HOLOLENS makes it easier for mentoring to be done by health professionals to novice medical students through the use of teleconferencing. It also allows medical professionals to get feedback from the team members and hence efficient medical operations. This reduces burnouts that usually occur in medics due to the maximum attention required to attend several patients. 3D images that are displayed in the form of CT scans may also be displayed on a screen for further checkups and attention [14].

F. Limitations of HOLOLENS
Although HOLOLENS has been embraced in various fields of economy and technology, it still has several limitations that if not taken care of may lead to serious damages to patients or losses in businesses. The hardware limitations are the major ones that involve heavyweight, batteries with a shorter lifespan, poor visual system in terms of rendering of images, low speeds in the use of gestures among other minute limitations [14]. Considering that HOLOLENS is connected to the internet, the medical data, for instance, may be exposed to malicious software like viruses and persons like hackers. In some instances, the HOLOLENS has to kill processes or apps to protect its memory and hence may lead to disastrous consequences taking the case of a patient undergoing operations that require maximum attention [15] [16].

IV. PROPOSED ARCHITECTURAL METHOD
According to background research, HOLOLENS is an incredible holographic computer with great potential in technology. However, the device still needs some improvement in terms of architecture that comes in several forms, for example, hardware and software parts. The hardware may mainly deal with optimizing the components that will at the same time be compatible with the software. The software is to ensure that the functionality of the hardware is presented to the user in a platform efficient and easy to interact with. The following are the proposed architectural designs that should be made for HOLOLENS to be optimized and hence achieve its maximum potential. The main focus is the rendering techniques, ergonomics, and the best user experience as much as possible.
Increasing the light points for resolution boost in conjunction with the number of light concentrated per radian area for density improvement Fig. 2. Currently, HOLOLENS considers holographic density more than the light points which make the images brighter but the images seem blurred. Light points are created from a combination of colors mainly red, green and blue, the RGB colors. The more the light points, the higher the holographic density hence brighter images that are easy to interact with. Considering that holographic images come in 3D form, it means brightness is essential for good user experience. HOLOLENS was created to optimize a holographic density approximated to 2.5K radians and the light points to be applied were 2.3 million. Increasing the number of light points will increase the efficiency of HOLOLENS especially attacking robots which takes less coverage of the screen where HOLOLENS has more advanced functions such as gaze, spatial, sound and mapping design working by MRTK [17]. Holographic intensity from the HOLOLENS is also to avoid images getting blurred to the point where a user sees through the transparent lenses of HOLOLENS. This improves the sensitivity of the lens and detection of the user in the environment. The conversion of 2D to 3D together with the detection of voice and data is recorded as shown in the flowchart below Fig. 3 and Fig. 4.
Lenses are what magnify the images to be interacted with while the field of view determines the range or area that the eye can see or interact with images. Increasing the field of view of HOLOLENS lenses while at the same time maintain the quality of images displayed will be an optimizing step to visualization. This means more hardware maneuvers and software reconstruction to go in line with the specification. Most users who have used HOLOLENS described the field of view as rectangular which means that what can be observed depends on the rectangular area alone. Adjusting the vertical view of the lenses will make increase the field of view giving the user a clearer vision as they interact with holographic images. The ability to immerse, where the user gets deep into the holographic world, is what the users demand considering the use in situations such as surgery or observation of tiny cancerous growths that require clearer vision alongside coupled with maximum care.
The field of view reported being rectangular which makes the user see fewer HOLOLENS can be improved by using remote middleware of the holographic HOLOLENS connected to the PC by the Unity engine which displays images in 3D. The PC with high performance takes most of the workload and renders the images, passes them to Wi-Fi connection then which is then received by the HOLOLENS in frames form. Since the workload has been reduced it just displays, captures, and transmits information interactively which includes gestures alongside voice. The Geographical Information System (GIS) [18] collects information from the environment in real-time. The holographic remote player adjusts the field of view accordingly depending on the scenes either large or small. The pictures below give a demonstration of this concept in Fig. 5. This is the behavior during smaller scenes. It can be noted that during the small scenes there is no need for a holographic remote player since the applications can be deployed directly to the HOLOLENS devices Fig. 6.
Adjusting the HOLOLENS to allow greater pupillary distance so that the user does not get eye strains or motion sickness which may come when the holographic computer is used for a long time. Users have complained that the pupillary distance is too short especially considering some users are long-sighted while others are short-sighted. When the HOLOLENS is moved to a considerable distance, the images get blurred and rendering problems set in. In some cases, if a user moves the HOLOLENS too close to the eye, the holographic images disappeared meaning the user is inconvenienced. Adjusting the pupillary distance to a considerable distance that can fit the nature of the user's eye will create a good experience for the user. Considering biometric techniques to the eye such that HOLOLENS will adjust according to the movement of the user's pupil and in conjunction with HOLOLENS lenses will ensure accurate and customized images to the user. www.ijacsa.thesai.org   Increasing the number of GPUs alongside the CPU and HPUs so that images are rendered much faster and with a lower workload to the motherboard of the HOLOLENS. The HOLOLENS has a motherboard that contains all the hardware meaning the space is limited. Unlike Oculus Rift, HOLOLENS is a computer that does not require to be mounted, that is, a standalone computer. This means the rendering of games may be quite difficult due to limited storage making some game lovers lose interest when their games fail to load as expected [10]. Increasing the number of GPUs with the HPU will ensure the rendering of quality images and videos. With the minimum motherboard space, it will require the minimization of the size of the hardware and at the same time maximizing the storage space. Fig. 7 below diagram shows how the GPU and CPU perform the tasks in HOLOLENS.
Increasing the number of FPS to the HOLOLENS will also optimize the images and data rendered to the screen. The frames per second of the HOLOLENS at the moment are 60 fps meaning when a user takes a preview of the image, it drops to around 40 fps hence the image appears blurred. If the GPU is increased together with the HPU tailored for the HOLOLENS, the fps will be possible with the increased processing power. Therefore, what should always be considered is the maximum number of frames per second that can be comfortably supported by the motherboard of the HOLOLENS. 80 fps can be comfortably supported with the increased GPU and HPU also with the more space created if the data is streamed to the cloud using Wi-Fi connectivity. Streaming images, video and data to the cloud using the IEEE 802.11ac Wi-Fi to create more space for user interactivity and image rendering [19]. All the other limitations like rendering, increasing the number of light points among others but with the limited space, the applications will often crash, to preserve the HOLOLENS mostly reported in the way HOLOLENS kills some activities when it reaches the extent of low storage to preserve itself. The limitations of storage discourage HOLOLENS usages in situations like hospital theaters where precision is compulsory and much attention demanded [20]. The strength that the HOLOLENS comes is its ability to connect to the Wi-Fi network meaning the data can be streamed to Microsoft Azure, the cloud platform by Microsoft. It will make HOLOLENS a device to be embraced by everyone since more storage space for CPU to perform tasks will mean quality holographic images, a large number of applications to be supported among other features. It may, however, be limited by cloud storage but Microsoft will have to make cloud storage affordable so that a large number of users can afford to enjoy the capabilities of improved HOLOLENS.

V. RESULTS
The resulting proposed HOLOLENS will be a computer that is agronomical such that the user finds it comfortable when using it. Creating more storage space by allocating space in the cloud to the data in the HOLOLENS will make it a reliable computer to be used in a wide variety of places without the fear of disastrous effects. Visualization in the HOLOLENS is more concerned with the brightness of the holographic images but with the proposed system, the clarity of the images is also put into account so that a user can easily immerse themselves in the composite world. Wi-Fi storage came with the HOLOLENS since the beginning but the strength of the Wi-Fi, when combined with the cloud storage in streaming the images, will tap its greatest potential. A final product with a clear vision, improved field of focus, adjustment of pupillary distance and improved image rendering will make HOLOLENS a device to be embraced by companies and investors Fig. 8.

VI. CONCLUSION AND FUTURE WORK
Indeed, HOLOLENS is a device that leads the computer technology to another era of augmented age. It is a fantastic device that has already been applied in some sectors though it is still at the testing stage. However, the HOLOLENS has many limitations which can be improved although some improvements may involve a lot of expenses for the company. Nonetheless, if the device is improved as proposed, the users will increase who will then increase the profit margin. The holographic computer has higher demand in institutions like health and education owing to its ability to immerse the user in a virtual world where they can retain a lot of information when compared to reading or watching a video. In the hospital operation room, especially for treatments like cancer which involves microscopic organisms, the HOLOLENS can combine the virtual reality with the real world giving the medical personnel the needed feedback from colleagues and efficient treatment of patients.

ABBREVIATIONS
 3D -Three Dimensions. The type of dimensional geometry involves length, width, and height, unlike 2D which consists only of length and width.
 AR -Augmented reality. The technology that advances VR in that the user not only intersects with a limited number of objects but a wide variety including geometrical operations like rotation among others. AR superimposes images created by the computer to the real world making the user have a composite view.
 C# -C sharp -A programming language that has the same characteristics as python and other objectoriented programming languages.
 CTcomputed tomography. This is a tomography that comes in the form of a computer controlling the motion of scans and x-ray detectors along with the source. CT scan takes images, processes them and give desired output in the form of a holographic image.
 eMMC -Embedded Multimedia Card -A technology that acts as storage for most mobile phones and related devices. On the phone, it is the internal storage that comes with the device apart from the external storage that someone can add up the storage.
 Fpsframes per second. The number of varying images or data that can be handled at once by a camera.
 GB-Gigabytes. The storage measurement capacity equivalent to approximately 1000MB.
 GPU-Graphics Processing Unit. The computer processor dedicated to rendering graphic images for quality graphics to be displayed.
 HPU-holographic processing unit. One of the dedicated processors tailored for processing images in holograms especially HOLOLENS. www.ijacsa.thesai.org  IEEE802.11institute of electrical and electronics engineers. It is the framework for the Wi-Fi network.
 LAN -Local area network. A network that gives connectivity to only a certain radius in an area, unlike the internet that covers the whole world.
 LPDD3 -Low Power Double Data Rate.
 MB-Megabytes. The memory measurement capacity equivalent to 1000 KB.
 MR -Mixed reality. Combination of AR with realworld objects making objects look as though they are real.
 OOP -Object-Oriented Programming. The type of programming that employs the use of objects instead of structures. OOP separates the object from the model and view making it easier to upgrade anytime without a lot of readjustments.