VEHICLE LICENSE PLATE DETECTION AND RECOGNITION
Table Of Contents
<p>
TABLE OF CONTENTS </p><p>ACKNOWLEDGEMENTS ............................................................................................................... II </p><p>LIST OF FIGURES ...................................................................................................................... V
ABSTRACT .................................................................................................................................... VII</p><p>
Chapter 1
INTRODUCTION AND BACKGROUND ................................................................... 1 </p><p> 1.1 Research Topic and Objectives ................................................................................................. 1 </p><p> 1.2 Challenges ......................................................................................................................... 2 </p><p> 1.3 Background and Related Work ............................................................................................... 3 </p><p> 1.3.1 License Plate Detection ....................................................................................................... 4 </p><p> 1.3.2 Character Segmentation ....................................................................................................... 13 </p><p> 1.3.3 License Plate Recognition ..................................................................................................... 15</p><p> 1.4 Overview of Our Methods ...................................................................................................... 16 </p><p>2 LICENSE PLATE DETECTION ................................................................................................ 18</p><p> 2.1 Scanning Window ............................................................................................................... 18 </p><p> 2.2 HOG Features .................................................................................................................... 20 </p><p> 2.2.1 Feature Extraction Procedure (for Dense-HOG) .............................................................. 21 </p><p> 2.2.2 Implementation .............................................................................................................. 24 </p><p> 2.3 Support Vector Machine ......................................................................................................... 24 </p><p>2.4 Non-maximum Suppression ..................................................................................................... 26 </p><p> 2.5 Refinement of the Algorithm................................................................................................. 29
2.5.1 Edge Information.............................................................................................................. 29</p><p> 2.5.2 Scale Adaption..................................................................................................................... 30</p><p> 2.6 Results and Discussion ........................................................................................................... 30</p><p> 2.7 Summary ............................................................................................................................. 31 </p><p>3 LICENSE PLATE RECOGNITION ............................................................................................ 32 </p><p> 3.1 License Plate Alignment Using Color Information................................................................... 34 </p><p> 3.1.1 License Plate Alignment Without Angles.............................................................................. 34 </p><p> 3.1.2 License Plate Alignment With Angles............................................................................ 38 </p><p> 3.2 Plate Binarization Using K-means Clustering........................................................................... 40 </p><p> 3.2.1 K-means Clustering.............................................................................................................. 41 </p><p> 3.3 Character Segmentation Using an Innovative Histogram-based Model ................................. 43 </p><p> 3.4 Digit Characters and Capital Letters Recognition Using Simple Robust Features .................. 48 </p><p> 3.4.1 Using Bag-of-words Model: Voting Schemes........................................................................ 49 </p><p> 3.4.2 Using SVM Classifier.......................................................................................................... 53 </p><p> 3.5 Data set and Results ............................................................................................................ 54 </p><p> 3.6 Summary.................................................................................................................................... 57</p><p> 4 REAL TIME EMBEDDED SYSTEM ............................................................................................ 58 </p><p> 4.1 Hardware Part...................................................................................................................... 58 </p><p> 4.2 Software Part.........................................................................................................................59 </p><p> 4.2.1 The Main Board Side............................................................................................................. 59 </p><p> 4.2.2 The Child Board Side........................................................................................................ 60 </p><p> 4.3 Implementation..................................................................................................................... 62 </p><p> 4.3.1 Kernel Module................................................................................................................... 62 </p><p> 4.3.2 Main Board Program.......................................................................................................... 63 </p><p> 4.3.3 Socket and TCP................................................................................................................. 63 </p><p> 4.3.4 Child Board Program........................................................................................................ 64 </p><p> 4.4 Results and Discussion.................................................................................................................................... 64 </p><p>5 CONCLUSIONS AND FUTURE WORK ................................................................................. 66 </p><p>REFERENCES .............................................................................................................................. 67 </p>
Thesis Abstract
Thesis Overview
<p><b>1.0 INTRODUCTION AND BACKGROUND </b></p><p><b>1.1 Research Topic and Objectives </b></p><p>License Plate Recognition (LPR) is a problem aimed at identifying vehicles by detecting
and recognizing its license plate. It has been broadly used in real life applications such as traffic
monitoring systems which include unattended parking lots, automatic toll collection, and
criminal pursuit [5].
The target of this thesis is to implement a vehicle retrieval system for a Chinese
surveillance camera, by detecting and recognizing Chinese license plates. It will be useful for
vehicle registration and identification, and therefore may further contribute to the possibility of
vehicle tracking and vehicle activity analysis. </p><p>The proposed method includes two main steps: </p><p>1) License Plate Detection: Using SVM classifier with HOG features based on a sliding
window scheme, scan possible regions detected by edge information, and obtain license plate
candidates. Then apply Non-Maximum Suppression (NMS) to finalize the plate locations. </p><p>2) License Plate Recognition: The detected license plate will be aligned first, after which
its pixels can be successfully clustered by k-means into two classes: background pixels , and the
foreground pixels, e.g., the pixels of the characters. The plate is segmented afterwards, into
character patches that will be recognized using SVM classifier individually. </p><p><b>1.2 Challenges </b></p><p>The first challenge is plate variation. The plate can be various in location, quantity, size,
color, font, occlusion, inclination, and plates may contain frames or screws [1]. The second
challenge is environmental variation which includes change in illumination and background.
Weather conditions, lighting conditions, and even camera conditions may contribute to the
difficulty of this problem.
For Chinese license plates, the font is fixed except vanity plates designed by individuals,
which is very rare. Fig.1.1 shows some Chinese license plates from several provinces. The
challenge from the font is relieved while all the other challenges like variation in plate location,
plate quantity, its size, color, occlusion, and inclination still remains. The weather condition
where the images in the dataset are captured is various too. Images are captured in the daytime as
well as at night; the weather can be sunny or it can be rainy.
<br></p><p>
<b>1.3 Background and Related Work </b></p><p>License plate recognition (LPR) system contribute to applications such as traffic
surveillance, traffic law enforcement, automatic toll collection, vehicle parking identification,
and vehicle access control in a restricted area. Typically, an LPR system is composed of License
plate detection (LPD), license plate character segmentation (LPS) and License plate character
recognition (LPR).
License plate detection is commonly the first procedure in a LPR system. It aims at
locating the license plate, which provides the following LPR procedure with accurate region
information. Instead of processing every pixel in the input image of the system, which is very
time consuming, license plate detection is a necessary process before license plate recognition.
Methods on LPD can be classified into such categories based on the color level: 1) Binary
Image processing; 2) Gray-Level Processing; 3) Color Processing; 4) Classifiers [3]. While
based on the information used to detect the license plate, the LPD methods can be categorized
into six classes: 1) using edge information; 2) using global image information; 3) using texture
features; 4) using color features; 5) using character features; 6) combing two or more features [1].
<br></p><p>
1.<b>3.1 License Plate Detection </b></p><p><b>A. Using Edge or Boundary Information.</b></p><p><b> a. Filters </b></p><p>In [6]-[9], Sobel filter is used to detect edges, by which the boundaries of license plates
are represented due to color transition between the license plate and the car body. Two horizontal
lines are located when performing horizontal edge detection, and two vertical lines are located
when performing vertical edge detection. The rectangle is fixed when two set of lines are located
both at the same time. While in 2003, F. Kahraman [2] et al. applied Gabor filters to detect
license plate regions which achieved a good performance when images are of a fixed angle.
Gabor filters are helpful in analyzing textures as they are sensitive to textures with different
scales and directions.
b. Edge Detection
Based on the intuitive that the license plate is of some shape, most likely rectangular,
whose aspect ratio is known, methods are commonly used to extract plates from all possible
rectangles. Edge Detection methods are such ones to find the rectangles [10]-[13]. Edges can be
detected only vertically or horizontally, and can be statistically analyzed to determine license
plate candidate regions [12]. A fast vertical edge detection algorithm (VEDA) was proposed in
[14] for license plate detection, which is declared to be faster than Sobel operator by around
seven to nine times
<br></p>