Determining adaptive thresholds for image segmentation for a license plate recognition system

A vehicle license plate recognition (LPR) system is useful to many applications, such as entrance admission, security, parking control, airport and cargo, traffic and speed control. This paper describe an adaptive threshold for image segmentation applied to a system for Malaysian intelligent license plate recognition (MyiLPR). Due to the different types of license plates used, the requirements of an automatic LPR system are rather different for each country. Upon receiving the input car image, this system (MyiLPR) detects and segments the license plate based on proposed adaptive threshold via image and blob histogram, and blob agglomeration, and finally, it extracts geometric character features and classifies them using neural network. The use of the proposed adaptive threshold increased the detection, segmentation and recognition rate to 99%, 94.98% and 90% correspondingly, from 95%, 78.27% and 71.08% obtained with the fixed threshold used in the originally proposed system. Keywords—adaptive threshold; image segmentation; license plate recognition; neural network; computer surveillance;


I. INTRODUCTION
Automatic license plate recognition (LPR) is an important research subject due to its many applications.For local authorities, LPR is required for the purposes of law enforcement, border protection, vehicle thefts, automatic toll collection and perhaps traffic control.For others, automatic LPR can be applied to gain access into housing areas, automatic parking control and as a marketing tool in large shopping complexes, as well as for surveillance.
LPR consists of five major steps: image capturing, image processing, segmentation, feature extraction and pattern classification.Usually, an LPR system includes targeted functions and specifications, such as a fast alphanumeric character recognition system.In addition, 24 hours non-stop operation is expected and the ability to issue alarm messages which are sent out after the recognition phase.
An important issue in image preprocessing for a real time application, like license plate recognition, is the selection of the best threshold value for image binarization.Choosing the wrong threshold can cause failure in either the detection or the segmentation process, as the thresholded image could either totally hide the required objects or cause distraction in the region of interest due to variable illumination.Several researchers have proposed different ways for selecting the best threshold.For example Yoshimori et al. [1] used a real-coded genetic algorithm and a so called hereditary threshold determination method, Kim et al. [2] used an enhanced neural network, and Lee et al. [3] used adaptive local binarization.Another technique referred to as the Otsu automatic threshold [4] has also been extensively used in LPR and other applications.In Malaysia, perhaps 95% of the vehicles' license plates are in the form of a single or a double line with normal fonts (Figure 1(a)).The other 5% are made of special fonts, such as those shown in Figure 1(b).The halo or illumination and unstandardized Malaysian plate issues have made license plate recognition continues to face big challenges [5]- [9].On 20th January 2015, the signing ceremony for memorandum www.ijacsa.thesai.org of agreement (MoA) between UKM Tech (M) Sdn.Bhd. and Synergy Saver (M) Sdn.Bhd. was held successfully.According to Synergy Saver CEO Mr. Shariffpudin Basiron; Up to date, the license plate recognition technology for Malaysian license plate cases are still an open issue.Our company purchased and tested several existing LPR products, yet we failed to achieve high accuracy rate.It is due to lack of enforcement towards unstandardized license plate format practice from road user.We believed that a collaboration between researchers (UKM) and industry will be able to solve the issues regarding Malaysia license plate recognition.
The objective of this paper is to propose an adaptive thresholding method for thresholding both the number plate from the car image and the individual characters within the number plate region, and to compare its results with those obtained by a fixed threshold, another state of the art thresholding method, as well as Otsu's automatic thresholding method, in relation to a Malaysian license plate recognition system.This paper is organized as follows.Section II reviews the state of the art in automatic threshold selection.Section III explains the proposed method and Section IV presents and analyses the experimental results.Finally, conclusions are presented in Section V.

II. STATE OF THE ART
Thresholding is a straightforward technique in transforming a gray scale image into a binary image that can facilitate the segmentation process.If h, is the input image, the value of the output image, ℘, at position (x, y), given a threshold Ω, is: Recent most popular global binarization used on license plate recognition is Otsu thresholding method [10]- [12] for its simplicity and fast performance.In addition, one of the most recently introduced methods for threshold determination known as adaptive contrast is also based on Otsu method [13].Meanwhile, Niblacks and Sauvolas methods become an alternative to solve shadows and illumination problem [14], [15] on license plate images.Global binarization methods are dependent on distinct image foreground and background such that the preprocessing and post-processing on binarization can contribute towards optimal results.Some recent work such deblurring [16], image filtering algorithms [17], [18] and super-resolution [19] are good example for preprocessing.Although post-processing approach are not popular compared to preprocessing, some recent work such dirt effect elimination [20] show decent result in character segmentation.
One of the most recent introduced methods for threshold determination is based on the local entropy [21].This approach has been widely used for measuring the local information content or uncertainty and the information content of a probability distribution [22], [23].First, the co-occurrence matrix of the image is constructed considering pairs of pixels that are either one below the other or one next to the other.For N grey level values, the co-occurrence matrix is N × N in size.Selecting a threshold value M divides the matrix into four quadrants (see Figure 2).The entropy of each quadrant can be computed and the threshold that maximises the sum of the entropies of the quadrants that correspond to the object and the background is selected.If p ij is an element of the co-occurrence matrix, we normalise it so that the sum of all values inside each quadrant is 1.
Assuming a bright object, the elements of the matrix in quadrant C (see figure 2) are divided by P O , and the elements of the matrix in quadrant A are divided by P B , where, Then the Shanon entropy, E O (M ) and E B (M ) for object and background respectively, is: The threshold that maximises the total entropy, is considered to be the optimal threshold.Fuzzy [24], excess [25] and honey bee optimization [26] has been incorporated into the method of threshold identification.The local entropy computation time is similar to that for Otsu's automatic threshold determination method.In this paper, we shall compare the results of the entropy-based and Otsu's methods with the results of a heuristic method presented in the next section for an LPR system.
In some cases, a single threshold is not applicable for global segmentation.Hence, multiple threshold values are identified to segment an image globally.A statistical recursive approach proposed in [27] calculates first the mean μ and standard deviation σ of the image.Two thresholds are then defined: where κ 1 and κ 2 are free parameters.
Pixels with value below the low threshold constitute one segment and pixels with value above the high threshold another segment.In the output image, each such pixels is assigned the average grey value of the segment to which it belongs.Pixels with values in between the two thresholds are thresholded further, in the same way, by calculating their mean and standard deviation, to form two new segments in the output image, carrying grey values equal to their respective means, and a central segment, which may be further subdivided in the same way.The method is repeated recursively, until the central segment diminishes, or a prespecified number of thresholds has been defined, or the Peak Signal to Noise Ratio (PSNR) reaches a certain value.PSNR computes noise introduced in the image by replacing the true grey values of the pixels with the average grey values of the segments to which they belong: where

III. THE PROPOSED METHOD
The proposed thresholding method consists of three steps: (1) identify the type of image according to its histogram; (2) calculate blob distributions for various threshold values and (3) select threshold value(s) based on a simple heuristic decision rule.Each of the steps is explained next.

1) Identify the type of image according to its histogram:
At this stage the contrast of the image is assessed and the image is categorised as dark, of medium brightness or bright, according to the shape of its histogram.We simply partition the 8-bit image into three levels by dividing 256 into 3 predetermined categories and sum those pixel frequencies for each category.Hence, we used the following heuristic rules to assess the shape of the histogram of the image: Here ρ g is the number of pixels in the image with grey value g and the image is assumed to be an 8-bit grey image.The histograms of images 'ADG676', 'ACP92' and 'ACN65' (Figure 3) are shown in Figure 4.

2) Calculation of blob distributions for various threshold values:
We transform source image into binary images using as a series of 25 thresholds, starting from threshold 0 and increasing its value in steps of 10.For each threshold, the number of the connected components created is identified.In Figure 5 we plot the number of the connected components identified this way for each of the example images of Figure 3, versus the threshold used.We then consider the thresholds that created the peaks in these graphs.This sub-process determines the peak as the adaptive thresholds as the more number of blobs gained, therefore the higher potential for the important objects (foreground) to be segmented or separated from the background of an image.
3) Selection of the thresholds: Some preliminary experiments showed that if the type of image is of 'medium brightness', a threshold of 130 sometimes leads to good results.So for this type of image, threshold 130 is always used in addition to the other thresholds identified as explained next.For all types of image, all thresholds that correspond to the peaks of the blob distributions had to be used.Open circles identify the peaks of the blob distributions as in in Figure 5, and dictates in a listing as in Table I, for the example images of Figure 3. TThese thresholds produce images as shown in Figure 6.
We notice that in Figure 6(a) and 6(c), which correspond to the images in the category of 'dark', and 'bright', respectively, there is a clear character separation in the number plate for thresholds Ω Select = 40 for the 'ACN65' image, and Ω Select = 60, 90, 110, 170 for the 'ADG676' image.So, in the subsequent steps, the thresholds selected at this stage will be used to identify all white blobs.The flow chart of the proposed method for threshold selection is shown in Figure 7.

A. Blob Agglomeration
Further processing will only consider all white blobs identified by the selected threshold applied to an image.Hence, the minimum enclosing rectangle will represent each blob.Eventually, we remove noise by considering each blobs' height and width at a certain size.In an LPR operational system, the scale of the images captured is assumed to be known.So, these size thresholds may easily be selected and fixed from the beginning.In our case, blobs narrower than 5 pixels and shorter than 15 pixels were treated as due to noise and removed.Then, the agglomeration process selects the remaining blobs into several clusters, starting from the top left, corner of the top left blob and agglomerates blobs that did not differ very much in their vertical position in the image will build up the cluster, as well as in height.In particular, if Y current is the vertical coordinate of the top left corner of the minimum enclosing box of the cluster, and Y is the vertical coordinate of the top left corner of a blob, and H current is the height of the minimum enclosing box of the cluster, the new blob is added to the cluster if, where H is the height of the blob and α is some parameter (See Figure 8).
The cluster with the maximum number of blobs is considered to be the identified number plate.Then we consider the blobs in the cluster, one at a time.The system as explained in [28] will identify the characters inside the blobs.

IV. EXPERIMENTAL EVALUATIONS
The proposed thresholding method was tested within the LPR system presented in [28]- [30].This system uses geometric features [28], [30] and either a back-propagation multilayer perceptron (MLP-BP) or a support vector machine (SVM) to classify the characters.In this paper we use only the version with MLP-BP as it gave better results than SVM.This system uses Opencv library support to implement the thresholding method at its back end.While at its front end a fixed threshold (equal to 130) interface also provided to binarise the captured image [29].In this paper we supplement the front end of the system with automatic selection of the threshold using 1) the local entropy method [21]; 2) Otsu's method [4], 3) Multi level method [27], 4) the proposed method, and compare the results obtained with those of the original system.CAIRO and CAIT members collected a dataset consist of 1216 images and it was used in the experiments.
The experiments were performed using the dataset on Intel Dual Core PC with 1.33Ghz processor speed.If all clusters identified in an image consist of a single blob, then we say that no number plate was identified in this image.The first row of Table II shows the percentage of images in which the number plate was not identified.Note that at this stage we do not consider cases where a number plate region had been found but it was totally wrongly placed.These cases will be identified only at the character recognition stage.
Of the clusters that are considered as candidate number plates, we consider first whether the number of blobs corresponds correctly to the true number of characters in the number plate.If it does, we consider that the segmentation was right.Table II also lists the percentage of images for which one, two, or more characters were not segmented, as well as the cases when more blobs than characters in the number plate were identified.Finally, we report the results of the recognition stage.Of the correctly segmented blobs, Table III gives the percentage of wrongly recognised characters for each of the tested methods.The same results are presented also in Figure 9.
Finally, the overall performance of the LPR system is reported in Table V, for the various thresholding methods, for license plate.The 1216 images in the dataset includes 7835 characters in the number plates.Various schemes tested shows the percentage of correctly segmented characters and the percentage of the correctly recognized characters from those segmented are given in the first and second rows of Table IV.
However, the recognition of full number plates was one of the criteria judged for the performance of an LPR system.A number plate is recognised if all its characters are fully segmented and all are correctly recognised.The percentages www.ijacsa.thesai.orgaccuracy rates are calculated as follows [31]: ) Even though the fixed threshold [29] system has poor performance, it has the lowest recognition time (973.485ms)compared with the Otsu [4](3549.31ms),Local Entropy [21](2630.81ms),Multilevel [27] (3789.18ms)and the proposed threshold (4026.14ms).This is due to its lower complexity.www.ijacsa.thesai.orgSome results of success and failures of the Otsu threshold [4] LPR system are shown in Figure 11.Some results of success and failures of the fixed threshold [29] LPR system are shown in Figure 12.Some results of success and failures of the multilevel threshold [27] LPR system are shown in Figure 13.Examples of successful and failed experiments of the system using the proposed thresholding method are shown in Figure 14.
V. CONCLUSIONS Specific applications have their own peculiarities.License plate recognition systems are commercially available nowadays.However, generic LPR systems are not appropriate for all countries, due to variations in the lay out and the font of the characters used in the number plates.A Malaysian LPR system developed recently uses a fixed threshold to segment the number plate and the characters.In this paper, we showed that with the help of a taylor-made thresholding method we can increase the performance of the algorithm significantly.However, the proposed approach is computationally intensive and it cannot be used in its present form for a real time application.It may, however, be used for off-line processing of images with license plates.Another advantage of the proposed approach, is that the adaptive threshold values can adapt to the environment where the system is to be used, for example when there is a high or low contrast situation, such as during the night, mid-day, underground and in a rainy day.www.ijacsa.thesai.org

Fig. 1 :
Fig. 1: (a) Samples of common license plates and (b) Samples of special license plates in Malaysia.

Fig. 2 :
Fig. 2: A threshold M partitions the co-occurrence matrix into four quadrants.The maximum entropy threshold maximises the entropy in quadrants A and C. For a bright object, quadrant C corresponds to pairs of object pixels.

Fig. 6 :
Fig. 6: Examples of the thresholding results using as thresholds the grey values that correspond to the peaks of the graphs in Figure 5 for (a) 'ADG676', (b) 'ACP92' (also includes the result for the 130 threshold value), and (c) 'ACN65', respectively.

Fig. 8 :
Fig. 8: The blob on the right is added to the cluster on the left, only if u + v ≤ α × H current .

Fig. 9 :
Fig. 9: Error Analysis for Local Entropy, Otsu, Fixed, Multilevel and the proposed framework in the (a)segmentation, and (b)classification phases.

TABLE I :
The thresholds that create the peaks in the distributions of Figure5.

TABLE II :
Segmentation results.

TABLE III :
Classification results with MLP-BP.

TABLE IV :
Blob error analysis for Local Entropy, Otsu, Fixed, Multilevel and the proposed framework based on correctly segmented and correctly recognised characters.
of characters in the number plate.Finally, the percentage of correctly identified number plates (LPR) includes only those images for which the full car number was recognised.These results are presented in Table V and in Figure10.Table V andFigure 10 are constructed based on three types of accuracy rate: LPD, LPS and LPR.The LPD, LPS and LPR

TABLE V :
LPD, LPS and LPR rate for Local Entropy, Otsu, Fixed, Multilevel and the proposed framework.