Application of Image Processing Techniques for Tv Broadcasting of Sporting Events

—In this paper, we describe a system solution thanks to which virtual graphics, the projection of advertising images, logos, match scores and the distance measurements of players on the field may be overlaid on the plan of the different types of sports fields of real tested images. This solution relies on a study of the artificial vision and the Augmented Reality applied to TV broadcasting of sporting events where we have as input the original image to be processed, the image to be projected and the coordinates of the overlay position of the objects on the plan of the field. As an output, we have the overlaid objects in the processed image at the selected position in a more realistic way and in the background.


INTRODUCTION
Augmented reality is one of the fields of artificial or computer-assisted vision.It is a rapidly growing research area thanks to its underlying principle allowing the mixing of the real and virtual worlds [1], [2].
Thus, the objective of the research work carried out in the field of augmented reality has so far been primarily focused on the positioning in real time of virtual objects in a real scene [3].
To this end, appropriate measurement tools are needed.So is an adequate processing environment dedicated for the processing of images and information [4].The work we describe in this article purports to provide a solution which would enable to overlay graphics on the plan of a soccer pitch, tennis, handball or basketball courts, by embedding for instance, adverts, flags, match scores or the analysis of the match or the measurements of the distance of the players in relation to the goal on the field.
To do so, we need to make sure that the insertion of the objects on the field occurs in the most real way and under the objects in the foreground (players, tennis net, etc.)The combination of methods used has enabled us to reach a high level of reliability and robustness as far as the obtained results on the tested real images are concerned.

II. REVIEW ON ATIFICIAL VISION AND AUGMENTED REALITY
We present some studies on the artificial vision and augmented reality that have been presented in various researches: Researchers Seong-Oh & al. [5], have proposed a new mobile augmented-reality system that will address the need of users in viewing baseball games with enhanced contents.The overall goal of the system is to augment meaningful information on each player position on a mobile device display.To this end, the system takes two main steps which are homography estimation and automatic player detection.This system is based on still images taken by mobile phone.The system can handle various images that are taken from different angles with a large variation in size and pose of players and the playground, and different lighting conditions.They have implemented the system on a mobile platform.The whole steps are processed within two seconds.
Researchers Stricker, D. & al. [6], have presented several results from the research department "Augmented Vision" of the German Research Center for Artificial Intelligence.The driving idea of this work is to move from traditional Augmented Reality (AR) systems, which are often limited to visualization and tracking components, to AR cognitive systems, which have or gradually build knowledge about the situation and intentions of the user.Such systems will basically be much more unobtrusive and adapt the information presentation to the users' actual needs.To reach this goal, strong progress must be done in several areas, starting with 3D scene digitalization and analysis, body modeling and motion capturing, and action and workflow recognition.An overview of current results and work-in-progress of the Augmented Vision group in those areas is presented and finally discussed.www.ijacsa.thesai.orgResearchers Rui & al. [7], have discussed the vision-based registration techniques for augmented reality (AR) systems which have been the subject of intensive research recently due to their potential to accurately align virtual objects with the real world.The downfall of these vision-based approaches, however, is their high computational cost and lack of robustness.To address these shortcomings, a robust pose estimation algorithm based on artificial planar markers is adopted.This algorithm solves the problem of camera pose ambiguities and is able to draw a unique and robust solution.Experiments show the robustness and effectiveness of this method in the context of real-time AR tracking.
Researchers Lourakis & al. [8], have presented in their work the camera matchmoving witch is an application involving synthesis of real scenes and artificial objects, in which the goal is to insert computer-generated graphical 3D objects into live-action footage depicting unmodeled, arbitrary scenes.This work addresses the problem of tracking the 3D motion of a camera in space, using only the images it acquires while moving freely in unmodeled, arbitrary environments.A novel feature-based method for camera tracking has been developed, intended to facilitate tracking in online, timecritical applications such as video seethrough augmented reality and vision-based control.In contrast to several existing techniques, which are designed to operate in a batch, offline mode, assuming that the whole video sequence to be tracked is available before tracking commences, the proposed method operates on images incrementally, as they are being acquired.
Researchers Ji Hoon Choi & al. [9], have introduced a method for personalized data broadcasting service using TVA metadata.Appropriate metadata structure for personalized data broadcasting is explained by comparison with package metadata and data broadcasting contents.New scenario and contents structure for personalized data broadcast is described.Therefore the system and data flow mechanism for personalized data broadcasting service is presented.In fact, the number of broadcasting channels and contents are increasing with the arrival of digital broadcast and various broadcasting medium.However, there is a limit on searching of the program by using conventional program guide.

III. PRESENTATION OF THE PROPOSED SOLUTION
The implementation process of our solution consists mainly of three stages (See Figure 1):  Pre-processing which aims at filtering the colors in the background of the image.
 Processing which combines the computational methods of the Homography matrix corresponding to the image selected and the insertion of the graphics projected on the field plan.
 Post-processing which presents the display of the outcome image.

A. Stage 1: Background color filtering method
This method consists in eliminating the background color of the field of the selected image.some background colors should just be selected to use them later to carry out the filtering whose aim is to detect, for instance, the players on the field and the net (in the case of a tennis court) and all the objects that are in the foreground [10].
The Algorithm Background color Filtering Method: Select the position of the colors to use for filtering (from one to four entrance points) the background of the field of the selected image.
2) For i to 1 over the width of the selected image.
For j to 1 over the length of the selected image If the pixel color of the position (i, j) of the image is close to one the colors selected at the beginning Then it is not to be put into the filtered image

If not
Put the color of this pixel at the position (i, j) in the filtered image.

End If
End For End For 3) Display and save the filtered image to use it later

B. Stage 2: Computational method of the Homography matrix
The computation of the Homography matrix is the second stage in our solution thanks to which we will be able to calculate the matrix of the perspective correction and the www.ijacsa.thesai.orgliaison between the selected image and the corresponding field image, depending on the type of field (tennis, soccer, Handball, Basketball) so that we can use it later during the overlaying of the objects on the plan of the field which would represent the overall perspective projection matrix of the four points of the field and their four corresponding points on the original image of the selected field.
When plan objects project themselves on a captor, the images seen from different perspective points (straight or oblique) are linked through a projective transformation called " homography" of the form: P' = H P with each point P(x, y) having its corresponding P'(x', y').
Homography «H» is characterized by a 3x3 homogeneous matrix.This transformation induces a scale factor and includes only 8 independent coefficients (h33=1) [2].
With w and w' being the homogeneous coordinates.
To adjust an image, we need to determine the 8 coefficients of the H transformation which brings the points of the plane source image to a benchmark position.The points P and P' are linked by: With the 4 points: P0(x0,y0), P1(x1,y1), P2(x2,y2) et P3(x3,y3) of the source image being assigned to positions P'0(x'0,y'0), P'1(x'1,y'1), P'2(x'2,y'2) and benchmark P'3(x '3,y'3), W have de  Hence, with these 4 control points, we obtain the following system: The resolution of this linear system will enable us to calculate the matrix of the H perspective remedial Homography as well as these 8 coefficients [11].

Algorithm of the homography matrix computation method :
Select the type of field of the selected image 2) For i from 0 to 3 (with i being the chosen point index ranging from 0 to 3)  Recording of the (xi,yi) coordinates on the side of the selected Pi point in the «coor_cot» matrix of the selected image coordinates such that coor_cot(i,0) = xi and coor_cot(i,1) = yi.
Recording of the (x'i,y'i) side coordinates of the corresponding P'i point in the «coor_originale_cot» matrix of the original coordinates of field image such that coor_originale_cot(i,0) = x'i and coor_originale_cot(i,1) = y'i.End for 3) Transformation of the «coor_cot» and «coor_originale_cot» matrices into 2 «cord» and «cordorign» tables of size equal to 8 : 4) The filling-in of the 8*8 "T" Matrix according to the method [3] from the coordinates of the selected « cord» and «cordorign» tables in order to use them later in the computation of the Homography «H» matrix coefficients :  For j From 0 to 6 (not = 2) (with j being the index of Table T rows)

5)
Computation of «T -1 » matrix which represents the inverse matrix of the «T» matrix and then multiply it by the «coor_originale_cot» table so as to eventually have the 8 coefficients of the «H» matrix and fill them in an «Hcoef» size 8 table (being a product of a matrix and a vector):  For l from 0 to 7  For c from 0 to 7 The filling-in of the final Homography matrix "H" of a 3*3 size with the 8 computed coefficients with H(3,3)=1 and display it afterwards :  [Hcoef(3), Hcoef(4), Hcoef(5)],  [Hcoef (6), Hcoef (7), 1 ]}

C. Stage 3: Method of overlaying graphics on the field's plan
This method makes it possible to superimpose graphics on the field's plan in accordance with a selected position and according to various choices such as the projection of a selected image on the field of the processed image regardless of the nature of the selected image, (the advertising image, flags image or the product image) as well as the possibility of inserting circles fr the for the follow-up of the players on the field by giving the radius of the circle and an off-side line for a given player ( in the case of soccer) which will be projected on the field of the processed image and the insertion of a distance arrow projected on the field of the processed image (in the case of a soccer or handball field) by measuring the distance in meters separating the the goal ( right or left) from a chosen position and drawing up this arrow projected on the field according to the dimensions in meters of the selected original field.
In addition, this method provides the possibility of writing a statement or the score of the match for the objects projected directly on the plan of the field of the selected image depending on the selected position.2) Insert the selected graphic into the field's original image corresponding to the position (x'p, y'p):

Algorithm of the method of graphics overlay (chroma keying) on field's plan
-In the Case of the integration of a selected image to be projected , we have used the summation of the pixels of the two images, choosing a given degree of transparency.
-In the case of the projection of a circle on the field's plan, we declare a graphic from the field's original image and draw a circle having a radius computed in pixels according to the following formula:

Radius size in pixels = ((Original field's width in pixels * Size of the radius entered in meters) / Original field's width in meters )
-In the case of an insertion of a distance arrow projected on the field, we declare a graphic from the original field's image and draw on this graphic an arrow having its starting point at the position (x'p, y'p) and arrival point at the position (0, (Image's height « img_orign ») /2) in the event of the arrow pointing towards the left side of the Goal, otherwise the arrival point shall be at the position (Image Width « img_orign », www.ijacsa.thesai.org-In the event of an integration of an off-side line, a graphic is declared using the original field's image and an arrow is drawn on this graphic, having its starting point at the position (x'p, 0) and its arrival point at the position (x'p, Height of the original field's image).
3) Compute the matrix «H -1 » which represents the inverse matrix of the Homography matrix «H», then proceed to the multiplication of each point P' of the position (x',y') of the original field's image obtained through the «H -1 » matrix, the inverse of the homography, to have its corresponding point P of the (x, y) position on the image which is being processed so as to eventually secure the projection of the integrated graphic on the field's plan of the processed image according to the formula P = H -1 P' making sure that the objects which are in the foreground appear using the filtered image obtained through the first method defined for the filtering of the field's background ,i.e.putting only the pixels which do not exist in the filtered image.

4)
The processed image may be saved on a disk or the processing may be cancelled.

IV. RESULTS OF THE APPLICATION
In this section, we will show examples of field images which we have tested using our solution as well as other methods.With regard to the background color filtering, for each type of pitch, we provide three figures., the first two of which (Figures 2 and 3) give an example of an image filtering according to the method of selection of four background colors of the field image of our solution as well as that of the image of the obtained result.However, the third image (Figure 4) accounts for the image tested using the methods of the Emgu CV library: it is a matter of converting the image into a grey level and then apply to the image the predefined function «image.InRange (color1, color2) » which makes it possible to eliminate two random colors existing in the obtained image.Then, we apply the "Gaussian" filter in order to have the output image [12].We have made use of the same methods of our solution on tennis courts' images (Figures 5 & 6).
Figure 7 shows the third image which is the image tested according to the Emgu CV library methods.
In the remaining sections of this article, we will be presenting some examples of images which we have tested so as to have the homography matrix proper for each image.For every investigated example, we will be specifying four figures, the first two of which represent the list of the coordinates in pixels of selected points and their corresponding and equivalent points in the field's image.Regarding the other two images:  The first one illustrates the Homography matrix calculated thanks to our programmed algorithm of the used method without resorting to the « Emgu CV » library.
Figures 8, 9, 10 and 11 illustrate an example of a processed football pitch image.Here are examples of images which we have used for the test of the integration of objects on the field plan of each processed image.The aim is to place images of advertising posters (Figures 18 and 19) or flags, insert match scores, draw circles, measure distances of players and insert arrows pointing towards the goal or draw an off-side line on the field according to a chosen position in the case of a football pitch image (Figures 16 and 17).According to the earlier results of the background color filtering method, it is worth pointing out that the result of the image filtered according to our method turns out to be better than the result of the filtered field image tested by another method.Thus, it can be said that our background color filtering method is more accurate for the filtering of fields' background color since most of the methods fail to detect the players on the field or the tennis net ( See Figure 7) Moreover, there is a color noise on the field which could be accounted for by an incomplete background color filtering.The implementation time varies according to the efficiency of the algorithm and the features of the used processing machine.Hence, we have carried out a study on the computation and implementation time variation of our background color filtering method algorithm for different images tested with different colors in order to get the result illustrated by the curve in (Figure 20).According to the results of the examples of the field images tested thanks to the homography matrix computation, we notice that after the plotting of the points and their corresponding ones on the different fields, we find that the result of the Homography matrices obtained and computed by our algorithm (the case of Figures 10 and 14) is equivalent to the result of the Homography matrices obtained using the predefined «Emgu CV» library function (the case of Figures 11  and 15).Thus, the resulting matrices are the same.We are therefore led to conclude that instead of using the «Emgu CV» library, which suffers some usage problems and shortcomings such as some installation and utilization problems with VB.NET and mainly problems of incompatibility with the operating systems.Thus, we go for the use of our algorithm which makes it possible to give an efficient result at a very fast computation and implementation time (not more than a few milliseconds) whatever the size of the image and the specified points' coordinates.
According to the results of the images tested by the method of the integration of objects on different types of fields, we have noticed that the display of the objects on the field varies according to the homography matrix associated with the processed field's image and we have found that the display takes place in an efficient and more realistic manner, for instance, the results perspective projection on the field of the off-side line (the case of Figures 16 and 17) apart from the integration of advertising images which are projected on the field and under the players as well as under the net of the tennis court (the case of Figures 18 and 19).This makes sure that the integration of what is virtual on what is real of the resulting image is close to the reality.This provides one of the advantages of the augmented reality with the possibility of resizing or the applying of a 90° rotation to manage the orientation of the image projected on the field as well as its size.
Thus, we have tested a method which consists in improving the quality of projection of the result on the field or the integration of lines and circle on the field by applying the Gaussian filter with a 5*5 -size convolution mask which represents the best smoothing filter type of the image for the projection of objects and avoid the noise, if need be, in the image before its projection on the field (Figures 21 and 22).
The Gaussian function is given by: With σ being the standard deviation's parameter It should be remembered that, in general, a Gaussian filter with σ < 1 is used to reduce the noise and if σ is higher than 1, then the filter will be used, the purpose of which is to construct an image that we could use to make a customized "unsharp mask".Hence, the bigger σ is, the more marked the blur applied to the image will be [14].
Undoubtedly, the major shortcoming of this filter is that the calculation is floating point and not integer at the complexity level [15], [16].The performance of the integration computation and of the objects projection on the field time varies according to the size of the image to be processed (in the case of the insertion of a circle, an offside line or a distance measurement) and also depends on the size of the image to be projected but the efficiency of the algorithm is determined thanks to the rapidity of the calculation and implementation time.We present the result obtained through a curve which represents the calculation and implementation time in seconds, in relation to the sizes of the images to be projected in pixels with its variation.(Figure 27) As to the obtained result which represents the calculation and implementation time in seconds in relation to the sizes of the images processed in pixels with its variation, it is represented by a curve (Figure 28).

VI. CONCLUSIONS AND OUTLOOK
This work demonstrates that our solution has been designed as a system suggested for the augmented reality applied to the telecasting of sports events aimed at analyzing match images where advertising content projected on the real image field plan is inserted so as to eventually create an image superimposing live action and another one calculated from a field model selected in such a way that it integrates virtually with the real image.
It may be concluded that the methods used in our solutions are relevant.To start with, the filtering is more accurate and occurs in the four selected colors efficiently detecting the players and the tennis net on the court to position them in the foreground.Then, the homography matrix calculation is efficient for different types of fields (Football, Tennis, Handball, and Basketball) depending on the models of the fields used.Finally, the overlay of virtual objects on the real output images is the closest one to reality, which is in compliance with the rules of the augmented reality concept.
Our solution, on the other hand, has remained a 2D one, because we have made use of the projection of each object on the selected field's plan at each time.A third dimension Z needs to be included through the use of other Homography matrix calculation techniques and methods so as to be able to inject 3D-images (bottles).
It can also be envisaged to carry out a study considering the possibility of integrating the image distortion correction before processing through using a method which would enable to automatically determine the positions of the field's corners.
The use of this method leading to results deemed to represent the reality should be validated by data generated by real and representative observations so that the integration test occurs in video or in real time.

Fig. 1 .
Fig. 1.Structure of the proposed solution

(
Image Height « img_orign ») /2) Should the arrow be pointing towards the right side of the Goal, we compute the selected position's coordinates in meters x1 and y1 according to the following formulae: x1 = ((x'p * Original Field's Width in meters « img_orign ») y1 = ((y'p * original field's height in meters) / original field's Height « img_orign ») Then, we compute the distance « D » in meters for the projected image on the field according to the following formula: D = √((-')² + ( /  -')²) With « LTm » being the width of the field in meters and « HTm » being the height of the field in meters.

Fig. 2 .Fig. 3 .Fig. 4 .
Fig. 2. Example of the selection of the background color chosen for a football pitch filtering

Fig. 5 .Fig. 6 .Fig. 7 .
Fig. 5. Example of the selection of the background color chosen for a tennis court filtering

Fig. 8 .
Fig. 8. Example of the selection of the football image points and their corresponding ones on the football pitch Figures 12,13, 14 and 15 represent an example of a processed tennis court image.

Fig. 12 .
Fig. 12. Example of the selection of the football image points and their corresponding ones on the tennis court

Fig. 16 .Fig. 17 .
Fig.16.Insertion of a 9 meter-diameter circle and the distance measurement of the player with the integration of the result projected on the football pitch (example 1)

Fig. 20 .
Fig. 20.Implementation time evolution curve of the background color filtering method depending on the image size

Fig. 21 .
Fig. 21.Result of the objects' integration before the use of the Gaussian filter

Fig. 22 .
Fig. 22. Result of the objects' integration after the use of the Gaussian filter So as to secure the efficiency of this method, we have carried out tests on the Handball and Basketball fields' images (Figures 23, 24, 25 and 26).

Fig. 23 .
Fig. 23.Original Handball field image to be processed

Fig. 25 .
Fig. 25.Original Basketball field image to be processed

Fig. 27 .
Fig. 27.Curve of the implementation evolution time in relation to the size of the image to be projected on the field. www.ijacsa.thesai.org

Fig. 28 .
Fig. 28.Curve of the implementation evolution time in relation to the size of the field image to processed.