運用新式立體視覺演算法於面型及線型攝影機三維模型重建技術之研究

A Study on Three-Dimensional Model Reconstruction Technology for Area and Line Cameras Using a Novel Stereovision Algorithm

10.6840/cycu201700045

陳建宏

立體視覺 ； 機器視覺 ； 半全域匹配 ； 三維重建 ； 線型掃描攝影機 ； stereovision ； machine vision ； semi-global matching (SGM) ； three-dimensional reconstruction ； line-CCD camera

中原大學機械工程學系學位論文

2017年

博士

陳冠宇

繁體中文

自動化光學檢測(automatic optical inspiration, AOI)為整合影像處理、光學設備及自動化技術，對產品進行非接觸檢測，提供高速度、高效率、高穩定、高重複的線上檢驗，是現代自動化生產不可或缺的一環。然而現今AOI檢測技術大多為二維平面影像的應用，可預期的是：當電腦硬體計算效率越來越好，對待測物的檢驗將不再僅限於平面位置，會需要對待測物的曲面深度有全面的感知能力，因此，基於三維影像的AOI系統必會愈趨普及。此外，面型攝影機普遍應用於現今的AOI系統進行影像擷取與分析，但隨著工件尺寸及精度等級一再向下突破，AOI系統的精度及處理速度也必須隨之升級，此時面型攝影機的解析度及取像速度即可能無法滿足生產線的要求，而線型攝影機因其高解析度及高速取像的特性相較於面掃描攝影機更具優勢，成為提升AOI系統的一種有效解決方案。因此，本文提出基於新式立體視覺演算法的線型攝影機之三維影像重建技術，以獲得更精確的三維影像模型。首先，本文分別建置由面型及線型攝影機構成的立體視覺影像擷取系統，再進行影像校正，最後結合新式立體視覺演算法及三種深度視差法：半全域匹配、區域局部匹配及影像切割演算法，分別進行三維影像模型的重建，藉由分析比較，找出最佳的三維影像模型重建方案。根據實驗結果，本文的主要成果有三：(1)以半全域匹配演算法為核心，利用前後各一組立體視覺攝影機模組進行拍攝，藉此重建目標物360度的三維模型；(2)將半全域匹配演算法得到的深度模型結合新式立體視覺演算法並對非均勻離散層級與仿射幾何空間進行誤差修正，得出更精確的深度數值；(3)利用線型攝影機模擬生產線中對目標物連續進行影像擷取，完成重建其三維模型。

In order to integration the image processing, optical equipment and automation technology on automated optical inspection. Provide high speed, high efficiency, high stability, and high repeatability is indispensable part of automated production on non-contact detection. In recent years, most AOI detection technology is about the application of 2D planar imaging. It is expected that: When the computer hardware computing efficiency is getting better, detect of the object will no longer be limited on plane, we need to measure the surface depth of object comprehensively, Therefore, the AOI system based on the three-dimensional detection will become popular. Area cameras are used in today's AOI system for image capture and analysis widely. However, the smaller workpiece size and higher precision grade are required, accuracy and processing speed of AOI system also need to be upgraded, resolution and capture speed on area cameras may not fulfill the requirements of the production line. To promote the ability on AOI system, using line camera could be an effective solution because of its high-resolution and high-speed Image capture. Therefore, this paper proposes a three-dimensional image reconstruction technique for linear cameras based on a novel stereovision algorithm to obtain more accurate three-dimensional image model. First, the paper construct the surface type and line type camera composed of stereo vision image capture system, respectively. Then correct left and right image. Finally, combined with novel stereovision algorithm and compare three kinds of depth matching method: Semi-Global Matching algorithm, Local Feature Matching, Grapth Cuts algorithm. Reconstruction three-dimensional model, respectively. Analyzing and compare the result, then find the best three-dimensional image reconstruction model. According to the experimental results, the results of this paper are as follows: The first we reconstruct the 360° three-dimensional model by two pairs CCD cameras shooting front and behind the target, and use Semi-Global Matching (SGM) algorithm compute the depth of target. The second combine Semi-Global Matching (SGM) algorithm with novel stereovision algorithms and use non-uniform spacing of discrete depth level error correction and affine space warping error correction to fix the depth of target. The third construct a detection system to simulate the production line and reconstruct the 3D model of target by two line-CCD camera

1. [1] Z. N. Li, and G. Hu, “Analysis of Disparity Gradient Based Cooperative Stereo,” IEEE Transactions on Image Processing, Vol. 5, No. 11, pp. 1493-1506, 1996.
   連結：
2. [2] D. Nitzan, “Three-Dimensional Vision Structure for Robot Applications,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 10, No. 3, pp. 291-309, 1988.
   連結：
3. [4] D. Marr and T. Poggio, “Cooperative computation of stereo disparity,” Science, Vol. 194, pp. 283-287, 1976.
   連結：
4. [5] S. T. Barnard and W. B. Thompson, “Disparity analysis of images,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. PAMI-2, No. 4, pp. 330-340, 1980.
   連結：
5. [8] R. Y. Tsai, “A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses,” IEEE Journal of Robotics and Automation, Vol. 3, No. 4, pp. 323-344, 1987.
   連結：
6. [9] A. Izaguirre, P. Pu, and J. Summers, “A new development in camera calibration: calibrating a pair of mobile cameras,” International Journal of Robotics Research, Vol. 6, No. 3, pp. 104-116, 1987.
   連結：
7. [11] W. Y. Yau and H. Wang, “Fast relative depth computation for an active stereo vision system,” Journal of Real-Time Imaging, Vol. 5, No. 3, pp. 191-204, 1999.
   連結：
8. [12] 李思漢, 基於匹配代價之非對稱立體匹配遮蔽偵測, 國立中央大學通訊工程研究所碩士論文, 2011.
   連結：
9. [13] 石家偉, 基於成本推算區域立體匹配之高解析度影像快速視差估計, 國立成功大學電腦與通信工程研究所碩士論文, 2013.
   連結：
10. [15] S. Sengupta, “Effects of unequal focal lengths in stereo imaging,” Pattern Recognition Letters, Vol. 18, pp. 395-400, 1997.
    連結：
11. [16] H. Ren and S. T. Wu, “Variable-focus liquid lens,” Optics Express, Vol. 15, No. 10, pp. 5931-5936, 2007.
    連結：
12. [17] K. Y. Chen, C. C. Chien, and C. T. Tseng, “Improving the accuracy of depth estimation in binocular vision for robotic applications,” Applied Mechanics and Materials, Vol. 284-287, pp. 1862-1866, 2013.
    連結：
13. [18] 陳冠宇、張永鵬、傅培耕，即時影像追蹤系統之設計與實現，中原學報，31期1卷，頁次21-29，2003。
    連結：
14. [20] 錢鉦津，模糊控制之立體視覺影像追蹤系統，碩士論文，中原大學機械工程學系，2005。
    連結：
15. [21] 陳冠宇、錢鉦津、張嘉麟、張永鵬，模糊控制之立體視覺影像追蹤系統，先進工程期刊，1期1卷，頁次21-29，2006。
    連結：
16. [22] 鍾宏英，多自主行動機器人無線感測網路室內定位系統之研究，碩士論文，中原大學機械工程學系，2014。
    連結：
17. [23] 傅培耕，即時物體追蹤之立體視覺導引自走車，，碩士論文，中原大學機械工程學系，2004。
    連結：
18. [24] 陳冠宇、張永鵬、傅培耕，具物體追蹤功能之立體視覺導引自走車，中原學報，33期1卷，頁次29-39，2005。
    連結：
19. [26] 張嘉麟，模糊邏輯控制可搭乘電梯之立體視覺伺服輪型機器人，碩士論文，中原大學機械工程學系，2005。
    連結：
20. [27] 蔡博仰，立體視覺伺服機器手臂之智慧追蹤控制的研究，碩士論文，中原大學機械工程學系，2007。
    連結：
21. [28] 劉易軒，視覺伺服擲球機械手臂之研究，碩士論文，中原大學機械工程學系，2010。
    連結：
22. [29] 謝宗仁，立體視覺導引擲球機器手臂之研究，碩士論文，中原大學機械工程學系，2008。
    連結：
23. [30] 曾彥博，立體視覺伺服雙目標追蹤系統之研究，碩士論文，中原大學機械工程學系，2009。
    連結：
24. [31] 陳冠宇、郭瑞濠、曾彥博、周晏德，立體視覺伺服雙目標追蹤系統之發展，先進工程期刊，6期2卷，頁次117-123，2011。
    連結：
25. [32] 郭瑞濠，立體視覺多目標追蹤系統之發展，碩士論文，中原大學機械工程學系，2009。
    連結：
26. [33] 錢鉦津，改良式立體視覺演算法應用於機器視覺系統之研究，博士論文，中原大學機械工程學系，2013。
    連結：
27. [43] O. Faugeras, E. Bras-Mehlman, and J. D. Boissonnat, “Representing stereo data with the Delaunay triangulation,” Artificial Intelligence, Vol. 44 No. 1-2, pp. 41-87, 1990.
    連結：
28. [45] S. Shen, “Accurate multiple view 3D reconstruction using patch-based stereo for large-scale scenes,” IEEE Transactions on Image Processing, Vol. 22, No. 5, pp. 1901-1914, 2013.
    連結：
29. [46] 蘇致融, 利用雙稜鏡拍攝於3D影像重建, 國立成功大學電腦與通信工程研究所碩士論文, 2010.
    連結：
30. [47] 蘇偉華, 基於LOD演算法之3D影像重建加速系統, 國立台北科技大學電腦與通訊研究所碩士論文, 2011.
    連結：
31. [48] 楊濟駿, 影像透明度分析於結構光影像解碼之應用與3D影像重建, 國立交通大學應用數學系碩士論文, 2012.
    連結：
32. [50] J Weng, P. Cohen, and M. Herniou , “Camera calibration with distortion models and accuracy evaluation,” IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol. 14, No. 10, pp. 965-980, 1992.
    連結：
33. [51] Tsai, R. “A Versatile Camera Calibration Technique for High-Accuracy 3D Machine Vision Metrology Using Off-the-Shelf TV Cameras and Lenses,” IEEE Trams. Robot. Automat, Vol. 3, Iss. 4, pp.323-344, 1987.
    連結：
34. [52] Bougnoux, S. “From projective to Euclidean space under any practical situation, a criticism of self-calibration,” In Proceeding of the 6th International Conference on Computer Vision, Vol. 4, Iss. 7, pp.790-796, 1998.
    連結：
35. [53] D. C. Brown, “Close-range camera calibration,” Photogrammetric Engineering, Vol. 37, pp. 855-866, 1971.
    連結：
36. [54] Kuan-Yu Chen, Cheng-Chin Chien, Wen-Lung Chang, Chi-Chung Hsieh, “Improving the Accuracy of Depth Estimation Using a Modified Stereo Vision Model in Binocular Vision”, The 10th International Symposium on Measurement Technology and Intelligent Instruments , Daejeon, Korea, 2011.
    連結：
37. [57] Daniel Scharstein, “ View Synthesis Using Stereo Vision,” Dissertation of Cornell University PHD, 1997.
    連結：
38. [59] Z. Zhang, “Flexible camera calibration by viewing a plane from unknown orientations,” Proceedings of the 7th International Conference on Computer Vision, Corfu, Sep. 1999, pp. 666-673.
    連結：
39. [60] H. Hirschmuller, “Stereo processing by semi-global matching and mutual information,” IEEE Transactions of Pattern Analysis and Machine Intelligence, Vol. 30, pp. 328-341, 2008.
    連結：
40. [61] Daniel I. Barnea, amd Harvey F. Silverman, “A class of algorithms for fast digital image registration” IEEE Transactions on computers, Vol. c-21, pp. 179-186, 1972.
    連結：
41. [62] V. Kolmogorov, P. Monasse, and P. Tan, “Kolmogorov and Zabih’s graph cuts stereo matching algorithm,” IPOL Journal, Vol. 4, pp. 220-251, 2014.
    連結：
42. [3] J. Weng, P. Cohen, and N. Rebibo, ”Motion and Structure Estimation from Stereo Image Sequences,” IEEE Transactions on Robotics and Automation, Vol. 8, No. 3, pp. 362-382, June. 1992.
43. [6] S. T. Barnard and M. A. Fischler, “Computation stereo,” Computing Surveys, Vol. 14, pp. 553-572, 1982.
44. [7] M. Z. Brown, D. Burschka, and G. D. Hager, “Advances in computational stereo,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 25, No. 8, pp. 993-1008, 2003.
45. [10] E. Grosso and M. Tistarelli, “Active/dynamic stereo vision,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 17, No. 9, pp. 868-879, 1995.
46. [14] 陳家泓, 根據極幾何與視差之適應性匹配增強本體移動估測, 國立高雄大學資訊工程學系碩士論文, 2013.
47. [19] 應福仁 ，DSP即時立體影像系統開發與移動目標物量測之應用，碩士論文，台灣科技大學機械工程學系，2007。
48. [25] 謝承哲，移動機器人立體視覺測距及追蹤移動目標，碩士論文，台灣科技大學電機工程學系，2009。
49. [34] R. Khilar, S. Chitrakala, and S. Selvamparvathy, “3D image reconstruction: techniques, applications and challenges,” International Conference on Optical Imaging Sensor and Security, Coimbatore, Tamil Nadu, India, Jul 2-3, 2013.
50. [35] A. S. Malik, T. S. Choi, and H. Nisar, Depth Map and 3D Imaging Applications: Algorithms and Technologies, Hershey, PA, USA: Information Science Reference, 2012.
51. [36] S. M. Seitz, B. Curless, J. Diebel, D. Scharstein, and R. Szeliski, “A comparison and evaluation of multi-view stereo reconstruction algorithms,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition, New York, USA, Jun. 17-22, 2006, Vol. 1, pp. 519-526.
52. [37] S. M. Seitz and C. R. Dyer, “Photorealistic scene reconstruction by voxel coloring,” International Journal of Computer Vision, Vol. 32, No. 1, pp. 151-173, 1999.
53. [38] A. Treuille, A. Hertzmann, and S. M. Seitz, “Example-based stereo with general BRDFs,” 8th European Conference on Computer Vision, Prague, Czech Republic, May 11-14, 2004, Vol. 2, pp. 457-469.
54. [39] T. Fromherz and M. Bichsel, “Shape from multiple cues: Integrating local brightness information,” 4th International Conference for Young Computer Scientist, Beijing, China, Jul. 19-21, 1995.
55. [40] K.Kutulakors and S. M. Seitz, “A theory of shape by space carving, International Journal of Computer Vision, Vol. 38, No. 3, pp. 199-218, 2000.
56. [41] R. Szeliski, “A multi-view approach to motion and stereo,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Fort Collins, Colorado, USA, Jun. 23-25, 1999, Vol. 1, pp. 157-163.
57. [42] V. Kolmogorov and R. Zabih, “Multi-camera scene reconstruction via graph cuts,” 6th European Conference on Computer Vision, Dublin, Ireland, Jun. 26-Jul. 1, 2002, Vol. 3, pp. 82-96.
58. [44] C. J. Taylor, “Surface reconstruction from feature based stereo,” 9th IEEE International Conference on Computer Vision, Nice, France, Oct. 13-16, 2003, pp. 184-190.
59. [49] R. C. Gonzalez, and R. E. Woods, Digital Image Processing 2/e, Addison Wesley Publishing Co., 1992.
60. [55] R. Hartley, and A. Zisserman, Multiple View Geometry in Computer Vision, Cambridge, UK: Cambridge University Press, 2006.
61. [56] 劉柏宏，從透視法到投影幾何，勤益科技大學通識教育中心，台灣通識網課程資料庫。
62. [58] 黃武雄，高中解析幾何後記，數學傳播季刊第五卷第一期，頁37-46，1981。