Method for 3 D Image Representation with Reducing the Number of Frames based on Characteristics of Human Eyes

Method for 3D image representation with reducing the number of frames based on characteristics of human eyes is proposed together with representation of 3D depth by changing the pixel transparency. Through experiments, it is found that the proposed method allows reduction of the number of frames by the factor of 1/6. Also, it can represent the 3D depth through visual perceptions. Thus, real time volume rendering can be done with the proposed method. Keywords—3D image representation; Volume rendering; NTSC image display


I. INTRODUCTION
Computer input by human eyes only is proposed and implemented [1]- [3] together with its application to many fields, communication aids, electric wheel chair controls, having meal aids, information collection aids (phoning, search engine, watching TV, listening to radio, e-book/e-comic/elearning/,etc., domestic helper robotics and so on [4]- [15].In particular, the proposed computer input system by human eyes only does work like keyboard as well as mouse.Therefore, not only key-in operations but also mouse operations (right and left button click, drag and drop, single and double click) are available for the proposed system.
It is well known that hands, fingers operation is much slower than line of sight vector movements.It is also known that accidental blink is done within 0.3 second.Therefore, the proposed computer input system by human eyes only decides the specified key or location when the line of vector is fixed at the certain position of computer display for more than 0.3 second.In other words, the system can update the key or the location every 0.3 second.It is fast enough for most of all application fields.
Meanwhile, 3D image display and manipulation can be done with 3D display.Attempts are also done with 2D display for 3D image display and manipulation [16], [17], on the other hands.Most of previous attempts are based on touch panel based manipulation by hands and fingers.As aforementioned, eyes operations are much faster than hands & fingers operations.Therefore, 3D image display and manipulation method by human eyes only is proposed in this paper.
3D image representations are widely used for a variety of applications such as medical electronics diagnostics image display, LSI pattern designs, and so on.Volume rendering is most popular method for 3D image representations, in general.It, however, takes a huge computational resources.
Volume rendering is in general costly.Real time representation, therefore, is not easy for volume rendering even for grid computing is used.For instance, 6 PCs of grid computing can be reduced by 35% of process time for the 3D representation with volume rendering.In order to reduce the process time of volume rendering, the number of frames which have to be display is reduced by using afterimage phenomenon.Time resolution of human eyes ranges from 50 to 100 ms.Illumination switching between on and off within the 50 to 100 ms is recognized as continuous illumination by human eyes.Therefore, it can be done to reduce the number of frames for 3D object image representation by using multilayer representation.On the other hand, refresh cycle of the NTSC video signal corresponds to 33 ms.Therefore, the interval of refresh cycle has to be within 33 ms for the proposed volume rendering.
The next section describes the proposed system followed by experiment.Then concluding remarks are described with some discussions.

A. Basic Idea
Basic Idea for the Proposed Method 3D model that displays a picture or item in a form that appears to be physically present with a designated structure.Essentially, it allows items that appeared flat to the human eye to be display in a form that allows for various dimensions to be represented.These dimensions include width, depth, and height.3D model can be displayed as a two-dimensional image through a process called 3D rendering or used in a computer simulation of physical phenomena.The model can also be physically created using 3D printing devices.Models may be created automatically or manually.The manual modeling process of preparing geometric data for 3D computer graphics is similar to plastic arts such as sculpting.
There are some previously proposed methods for 3D display such as tracing object contour and reconstruct 3D object with line drawings, and wireframe representation and display 3D object with volume rendering.There is another method, so called OCT: Optical Coherence Tomography.It, however, quit expensive than the others.Fundamental idea of the proposed method is afterimage.Response time of human eyes is that the time resolution of human eyes: 50ms～100ms.An afterimage or ghost image or image burn-in is an optical illusion that refers to an image continuing to appear in one's vision after the exposure to the original image has ceased.One of the most common afterimages is the bright glow that seems to float before one's eyes after looking into a light source for a few seconds.Afterimages come in two forms, negative (inverted) and positive (retaining original color).The process behind positive afterimages are unknown, though thought to be related to neural adaptation.On the other hand, negative afterimages are a retinal phenomenon and are well understood.Example of 3D image on to 2D display is shown in Fig. 1.This is the proposed system concept.In the example, "A" marked 3D image of multi-fidus which is acquired with CT scanner is displayed onto computer screen."B", "C", ... are behind it.

B. Displayed Image in Automatic Mode
Implementation of the proposed system is conducted.By using mouse operation by human eyes only, 3D image with different aspects can be recreated and display.It is confirmed that conventional image processing and analysis can be done with mouse operation by human eyes only.Fig. 2 shows the example of displayed layered images.In this example, 1024 of layer images are prepared for 3D object.By displaying the prepared layered images alternatively in automatic mode, 3D object appears on the screen.Furthermore, as shown in Fig. 2, internal structure is visible other than the surface of the 3D object.It looks like a semitransparent 3D surface and internal structure in side of the 3D objects.
Another example is shown in Fig. 3.In the figure, the first to 768 th layer images are shown together with a slant view of the 3D object image with the several layer images.1024 layer of images are created for the 3D object by using Open GL.Surface rendering with the 1024 layer of images makes a 3D object rendering like Fig. 3.

C. Changing the Transparency
In order to represent the depth information, transparency is used.The purpose of the depth information representation is to express the internal structure of the 3D object.By changing the transparency, somewhat depth information can be represented as shown in Fig. 6.In the figure, 100% denotes 0% transparent image while 5% also denotes 95% transparent image, respectively.It is clear that internal structure can be seen by changing transparency.Therefore, users can select the transparency depending on which portion of internal structure would like to see through visual perception.

D. Foggy Representation for Depth Representations
In order to represent the depth information, not only transparency but also foggy representation is added.Because of depth information representation is getting week depending on the transparency, it is needed to add some other representation for depth information.One of the effect is foggy representation with the different color.

E. Implementation
Implementation is done based on Open GL.Parameter setting can be done with the slide ruler shown in Fig. 8. Slid rulers for rotation angles in x, y, and z axis which ranges from 0 to 355 degree, refresh cycle of time interval which ranges from 1 to 100 ms, and object interval (the number of frames which ranges from 1 to 127) are available to set.The other parameter, such as transparency (ranges from 0 to 100 %), foggy representation can be set automatically.
As mentioned above, the time resolution of human eyes ranges from 50 to 100 ms.Illumination switching between on and off within the 50 to 100 ms is recognized as continuous illumination by human eyes.Therefore, it can be done to reduce the number of frames for 3D object image representation by using multi-layer representation.On the other hand, refresh cycle of the NTSC video signal corresponds to 33 ms.Therefore, the interval of refresh cycle has to be within 33 ms for the proposed volume rendering.Example of a frame image of the proposed volume rendering is shown in Fig. 9. 3D object image representation based on the proposed method with the image datasets used in this experiment were from the Laboratory of Human Anatomy and Embryology, University of Brussels (ULB), Belgium.Fig. 10 shows examples of the resultant images.The 3D images used are derived from CT scan images.In these cases, the number of frames used is 127 while the transparency is set at 50 %.Meanwhile, the refresh cycle of the time interval is set at 17 ms which corresponds to 58 f/s.Thus, 1/6 of reduction can be achieved successfully.IV.CONCLUSION Method for 3D image representation with reducing the number of frames based on characteristics of human eyes is proposed together with representation of 3D depth by changing the pixel transparency.Through experiments, it is found that the proposed method allows reduction of the number of frames by the factor of 1/6.Also, it can represent the 3D depth through visual perceptions.Thus, real time representation of 3D object can be displayed onto computer screen.
Further investigations are required for improvement of frame reduction ratio.The refresh cycle has to be optimized.Also, the transparency and foggy representation has to be optimized.In other word, a method for optimization of the parameters has to be created.

Fig. 1 .
Fig. 1.System Concept Such this layered 3D images are aligned with depth direction.Attached character "A" to "Z" are transparent and displayed at just beside the layered image at the different locations.Therefore, user can recognize the character and can select one of those characters by their eye.Arrow shows the line of sight vector.Curser can be controlled by human eyes only.By sweeping the character, 3D images are displayed by layer by layer.Therefore, it looks like time division delay of 3D images.

Fig. 6 .
Fig. 6.Examples of 3D object image representation with changing transparency Fig. 7. Examples of Fog added 3D image for depth representation Fig.7 shows examples of foggy representation of depth information representations.

Fig. 8 .
Fig. 8. Scroll slide bar for designation of parameters of 3D object image representations (the slide ruler is for the rotation angle in x axis followed by y and z axis and refresh cycle of the time interval of display as well as object frame interval ranges from 1 to 127

Fig. 9 .
Fig. 9. Example of a frame image of the proposed volume rendering III.EXPERIMENT

Fig. 10 .
Fig. 10.3D object image representation based on the proposed method with the image datasets used in this experiment were from the Laboratory of Human Anatomy and Embryology, University of Brussels (ULB), Belgium