結合基於單眼影像的車道線偵測與多層雷射的路緣偵測之可駕駛道路區域分析

Monocular Vision-Based Lane Detection and Multi-Layer Laser Based Curb Detection for Drivable-Region Analysis

10.6342/NTU201703689

吳余軒

車道線偵測 ； 道路地形邊界追蹤 ； 可駕駛區域偵測 ； 消失點偵測 ； 機率資料結合濾波器 ； 地圖匹配 ； Lane detection ； curb detection ； drivable region detection ； vanishing point detection ； integrated probabilistic data association filter ； map matching

國立臺灣大學電機工程學系學位論文

2017年

碩士

連豊力

英文

自動駕駛輔助系統是近年來相當熱門的主題，為了避免因為駕駛者沒有專心或是錯誤的判斷所導致的車禍意外，因此利用機器的人工智慧、電腦視覺以及影像處理技術的演算法配合安裝在車上的感測器來協助駕駛者的安全認知與判斷，在輔助系統當中，定位與感知是兩項重要的課題。 可駕駛區域分析是駕駛輔助系統對於感知的一項重要依據。在道路上，車子主要行駛於車道標線之內。然而，並非所有環境的道路都有車道標線可以遵循。因此，路面地形的邊界是另一項可駕駛區域的重要特徵，雖然多層雷射測距可以用來感知不同距離的道路邊界，但其結果容易受地形變化影響。此外，道路上的車道標線在影像上的延伸線所得到的消失點，可以用來提高反透視投影轉換的正確性。 定位是另一項輔助駕駛系統的核心之一，全球定位系統 (GPS) 是當前行車定位為不可或缺的技術。然而在複雜的動態環境中行駛，尤其大城市，GPS 多路徑反射的問題會很明顯。這樣得到的 GPS 定位信息很容易有幾米的誤差，因此必須藉助其他感測器來輔助定位。 本篇論文主要利用單眼相機得到的影像來擷取車道標線的資訊，透過初始消失點估測與反透視投影來協助近、遠距離的車道線偵測，本文的車道線偵測主要在高速公路、城市及校院區三個場景分別在不同條件環境，如: 照明不均、障礙物出現和車道的形狀，來進行偵測。在照明不均與轉彎的道路上，車道線皆可以被準確偵測，但在有障礙物出現的場景會有部分的錯誤，在短暫連續的直線的車道場景，車道線偵測的準確率可以達到 80%。 此外，利用多層雷射測距與機率資料結合濾波器 (IPDAF)來追蹤車道路面上地形的邊界，藉由疊代最近點演算法在偵測的地形邊界與已知的地形邊界地圖計算出均方根誤差，在地形邊界的準確率可以達到 80%。根據濾波器得到的追蹤機率，路面邊界的追蹤成功率在第一、第二層雷射可以達到 96%。接著透過座標轉換，結合影像座標上的車道線資訊與雷射極座標上的路面邊界資訊轉換到以車身為參考的平面座標來進行可駕駛區域的分析。最後，將多層雷射偵測得到的路面地形的邊界資訊視為道路上結構化的特徵，與高精度地圖匹配，來進行 GPS 偏移的修正。

Automatic driving assistance system has become a popular research in recent years. The algorithms based on knowledge of machine intelligence, computer vision, and image processing combining with sensors nstalled on advanced safety vehicle have been proposed to avoid collisions or accidents which are caused by the lack of recognition and miss judgement by driver. Localization and perception are two important issues for automatic driving assistance system. Drivable region analysis is one of the most important foundations for perception of driving assistance system. Generally, cars are driven on road between the both sides of lane markings. However, there is no any lane markings for references on nonstructural roads in some environments. Therefore, curbs are another road features for drivable region. Although road boundaries with different distance can be detected by multi-layer laser scanner, the results are affected by variation of terrain. In addition, a vanishing point provided by intersection of both sides of extending lines of lane markings in image can enhance the performance of inverse perspective mapping. Localization is another core of driving assistance system. Global positioning system, named GPS, is indispensable technology for currently driving positioning. While driving in complex and dynamic environments, especially downtown, it is obvious to suffer from the problem of multipath interference from satellites causing the meters drift of GPS. Thus, it is necessary to combine with other sensors for localization to correct the meters error of GPS. In this thesis, the information of lane markings are extracted based on image of monocular camera. Through the ways of vanishing point estimation and inverse perspective mapping, the proposed lane detection method can detect lane markings in near and far region. The experimental scenes for lane detection are classified into 3 categories which represent freeway, downtown, and campus in different conditions, including uneven illumination, obstacles on road, and shape of road. Lane markings can be detected precisely in conditions of uneven illumination and curve road, while there is partially false detection caused by obstacles on road in far region. In a sequence of continuous time steps, accuracy of proposed lane detection methods is high as 80% in scenes with straight lanes. Besides, multi-layer laser and integrated probabilistic data association filter (IPDAF) are used to detect curbs which are the boundaries of road surface. The results of curbs detection are evaluated by root mean square error of iterative closet point algorithm between curbs and curb map in prior with accuracy high as 80%. According to tracking probabilities provided by IPDAF, the accuracy of road boundaries tracking by layer 1 and layer 2 of multi-layer laser ranging near distance can be achieved to 96%. Then, in the way of coordinate transformation, the integrated information of lane markings in image coordinate and curbs on polar coordinate of laser is transformed to world plane coordinate where reference point is position of ego-vehicle for the propose of drivable region analysis. Finally, the curbs detected by multi-layer laser are considered as structural features on road. Using curbs as features and map matching with curb map in prior, the drift error of GPS can be corrected.

1. [15: Lin et.al 2010]
   連結：
2. Qing Lin, Youngjoon Han, and Hernsoo Hahn, “Real-Time Lane Departure
   連結：
3. Detection Based on Extended Edge-Linking Algorithm”, in Proceedings of Second
   連結：
4. [16: Ogawa & Takagi 2006]
   連結：
5. T. Ogawa and K. Takagi, “Lane Recognition Using On-vehicle LIDAR”, IEEE
   連結：
6. Intelligent Vehicles Symposium, Tokyo, Japan, June 13-15, 2006.
   連結：
7. [17: Wijesoma et al. 2004]
   連結：
8. 456-464, June, 2004.
   連結：
9. [18: Han et al. 2012]
   連結：
10. Road Boundary and Obstacle Detection Using a Downward-Looking LIDAR
    連結：
11. Sensor, IEEE Transactions On Vehicular Technology, Vol. 61, No. 3, March 2012.
    連結：
12. Lane-Detection System for Autonomous Vehicle Navigation in Challenging Road
    連結：
13. Scenarios”, IEEE Transactions on Vehicular Technology, vol. 63, pp540-555, Sep. ,
    連結：
14. Zhenqiang Ying and Ge Li, “Robust lane marking detection using boundary-based
    連結：
15. Acoustics, Speech and Signal Processing, Shanghai, China, March 20-25, 2016.
    連結：
16. [21: Gu et al. 2015]
    連結：
17. Conference on Pattern Recognition, Nov. 3-6, 2015.
    連結：
18. [22: Tan et al. 2014]
    連結：
19. curve lane detection based on improved River flow and RANSAC”, in Proceedings
    連結：
20. of IEEE 17th International Conference on Intelligent Transportation Systems,
    連結：
21. [23: Xu et al. 2009]
    連結：
22. fast and stable lane detection method based on B-spline curve”, in Proceedings of
    連結：
23. [24: Chen & Wang 2006]
    連結：
24. Qiang Chen and Hong Wang, “A real-time lane detection algorithm based on a
    連結：
25. Hyperbola-pair model”, in Proceedings of IEEE Intelligent Vehicles Symposium,
    連結：
26. [25: Jung & Kelber 2004]
    連結：
27. C.R. Jung and C.R. Kelber, “A robust linear-parabolic model for lane following”,
    連結：
28. in Proceedings of 17th Brazilian Symposium on Computer Graphics and Image
    連結：
29. [26: Pollard et al. 2011]
    連結：
30. Evangeline Pollard, Dominique Gruyer, Jean-Philippe Tarel, Sio-Song leng, and
    連結：
31. International IEEE Conference on Intelligent Transportation Systems, Washington,
    連結：
32. [27: Ozgunalp & Dahnoun 2014]
    連結：
33. Umar Ozgunalp and Naim Dahnoun, “Robust Lane Detection & Tracking based on
    連結：
34. International Conference on Acoustics, Speech and Signal Processing, Florence,
    連結：
35. Italy, May 4-9, 2014.
    連結：
36. [28: Bertozzi & Broggi 1998]
    連結：
37. generic obstacle and lane detection”, IEEE Transactions on Image Processing, Vol.
    連結：
38. [29: Tan et al. 2014]
    連結：
39. lane detection based on Improved River Flow and RANSA”, in Proceedings of
    連結：
40. IEEE 17th International Conference on Intelligent Transportation Systems,
    連結：
41. Weina Lu, Haifang Wang, and Qingzhu Wang, “A Synchronous Detection of the
    連結：
42. First International Workshop on Knowledge Discovery and Data Mining, Adelaide,
    連結：
43. Detecting Lane Boundary in Challenging”, Information Technology Journal, pp.
    連結：
44. 2300-2307, 2011.
    連結：
45. and Kalman Filtering-Based B-Spline Lane Tracking”, International Journal of
    連結：
46. Vehicular Technology, March 27, 2012.
    連結：
47. [34: Zhang et al. 2015]
    連結：
48. real-time curb detection and tracking method for UGVs by using a 3D-LIDAR
    連結：
49. sensor”, in Proceedings of IEEE Conference on Control Applications, Sydney,
    連結：
50. D. Musicki, R. Evans, and S. Stankovic, “Integrated probabilistic data association”,
    連結：
51. IEEE transactions on Automatic Control, Vol. 39, pp. 1237-1241, Jun. 1994.
    連結：
52. [36: Besl & McKay 1992]
    連結：
53. P.J. Besl and N.D. McKay, “A Method for Registration of 3-D Shapes,” IEEE
    連結：
54. Transactions on Pattern Analysis and Machine Intelligence, Vol. 14, No. 2, pp.
    連結：
55. [37: Fischler & Bolles 1981]
    連結：
56. illumination-invariant lane detection system”, in proceeding of Anti-Cyber Crimes
    連結：
57. [40: Gonzalez & Woods 2008]
    連結：
58. R. C. Gonzalez and R. E. Woods, Digital Image Processing, 3 rd ed., Editor:S. G.
    連結：
59. Miaou, Taiwan: Pearson, 2008.
    連結：
60. [41: Shaoiro et al. 2001]
    連結：
61. Linda Shapiro, 2001.
    連結：
62. Available:
    連結：
63. International Conference on Computer Research and Development, May 7-10,
64. 2010.
65. W.S. Wijesoma, K.R.S. Kodagoda, A.P. Balasuriya , “Road-boundary detection and
66. tracking using ladar sensing”, IEEE Transactions on Robotics and Automation, pp.
67. Jaehyun Han, Dongchul Kim, Minchae Lee, and Myoungho Sunwoo , “Enhanced
68. [19: Li et al. 2014]
69. Qingquan Li, Long Chen, Ming Li, “A Sensor-Fusion Drivable-Region and
70. 2014.
71. [20: Ying & Li 2016]
72. inverse perspective mapping”, in Proceedings of IEEE International Conference on
73. Jingchen Gu, Qieshi Zhang, and Sei-ichiro Kamata, “Robust road lane detection
74. using extremal-region enhancement”, in Proceedings of 3rd IAPR Asian
75. Huachun Tan, Yang Zhou, and Yong Zhu, Danya Yao, and Keqiang Li, “A novel
76. Huarong Xu, Xiaodong Wang, Hongwu Huang, and Keshou Wu, and Qiu Fang, “A
77. IEEE 10th International Conference on Computer-Aided Industrial Design &
78. Conceptual Design, Wenzhou ,China, Nov. 26-29, 2009.
79. Tokyo, Japan, June 13-15, 2006.
80. Processing, Curitiba, Brazil, Oct. 20, 2004.
81. Aurelien Cord, “Lane Marking Extraction with Combination Strategy and
82. Comparative Evaluation on Synthetic and Camera Images”, in Proceedings of 14th
83. USA, Oct. 5-7, 2011.
84. Novel Feature Extraction and Lane Categorization”, in Proceedings of IEEE
85. M. Bertozzi and A. Broggi, “GOLD: a parallel real-time stereo vision system for
86. 7, pp. 62-81, Jan., 1998.
87. Huachun Tan, Yang Zhou, Yong Zhu, Danya Yao, and Keqiang Li, “A novel curve
88. Qingdao, China, Oct. 8-11, 2014.
89. [30: Lu et al. 2007]
90. Road Boundary and Lane Marking for Intelligent Vehicles”, in Proceedings of
91. SNPD Eighth ACIS International Conference on Software Engineering, Artificial
92. Intelligence, Networking, and Parallel/Distributed Computing, Qingdao, China,
93. July 30- Aug. 1, 2007.
94. [31: Lu et al. 2008]
95. Weina Lu, Yucai Zheng, YuQuan Ma, and Tao Liu, “An Integrated Approach to
96. Recognition of Lane Marking and Road Boundary”, in Proceedings of WKDD
97. SA, Australia, Jan. 23-24, 2008.
98. [32: Tran et al. 2011]
99. Trung-Thien Tran, Hyo-Moon Cho, and Sang-Bock Cho, “A Robust Method for
100. [33: Lim et al. 2012]
101. King Hann Lim, Kah Phooi Seng, and Li-Minn Ang, “River Flow Lane Detection
102. Yihuan Zhang, Jun Wang, Xiaonian Wang, Chaocheng Li, Liang Wang, “A
103. Australia, pp. 1020-1025, Sept. 21-23, 2015.
104. [35: Musicki et al. 1994]
105. 239-256, Feb. 1992.
106. Martin A. Fischler and Robert C. Bolles “Random Sample Consensus: A Paradigm
107. for Model Fitting with Applications to Image Analysis and Automated
108. Cartography”, Communications of the ACM, vol. 24, pp. 381-395, Jun. 1981.
109. [38: Kortli et al. 2017]
110. Yassin Kortli, Mehrez Marzougui, and Belgacem Bouallegue, “A novel
111. 2nd International Conference, Abha, Saudi Arabia, March 26-27, 2017.
112. Books
113. [39: Laganiѐre 2011]
114. R. Laganiѐre, OpenCV 2 Computer Vision Application Programming Cookbook,
115. 1st Ed., Editor: Neha Shetty, Packt Publishing Ltd., May 2011.
116. G. Lindar, Shapiro, Strockman, and C. George, Computer Vision, 1st Ed., Editor:
117. Websites
118. [42: Iterative Closet Point Algorithm 2013]
119. Author: Jakob Wilm
120. https://www.mathworks.com/matlabcentral/fileexchange/27804-iterative-closest-po
121. int?requestedDomain=www.mathworks.com