Stereo Matching Using Synchronous Hopfield Neural Network

運用同步式霍普菲爾類神經網路求解雙影像對應問題

10.29977/JCIIE.200907.0005

孫德修(Te-Hsiu Sun)

雙影像對應 ； 雙影像系統 ； 同步式霍普菲爾類神經網路 ； 機器視覺 ； stereo matching ； correspondence problem ； synchronous Hopfield neural network ； computer vision

工業工程學刊

26卷4期（2009 / 07 / 01）

276 - 288

英文

深度資訊之取得為電腦視覺研究領域中之最要議題，而雙影像爲3D資訊取得技術之重要技術之一。本研究以同步式霍普菲爾類神經網路解決掃瞄式雙影性對應問題，首先，特性點以索伯運算子(Sobel operator)及自訂之臨界值於雙影像中分別取得，再將此資訊表示爲一雙影像對應最佳化問題之能量函數，此函數包含相異性、連續性、視差特性及單一性等條件，再以同步式霍普菲爾類神經網路求取此函數之最小植，最後再以一錯誤對應移除準則將錯誤之對應點移除。本研究以一般通用之雙影像作實驗驗證，實驗顯示本研究所提出方法可有效地解決該問題，並可應用於多種領域中。

Deriving depth information has been an important issue in computer vision. In this area, stereo vision is an important technique for 3D information acquisition. This paper presents a scanline-based stereo matching technique using synchronous Hopfield neural networks (SHNN). Feature points are extracted and selected using the Sobel operator and a user-defined threshold for a pair of scanned images. Then, the scanline-based stereo matching problem is formulated as an optimization task where an energy function, including dissimilarity, continuity, disparity and uniqueness mapping properties, is minimized. Finally, the incorrect matches are eliminated by applying a false target removing rule. The proposed method is verified with an experiment using several commonly used stereo images. The experimental results show that the proposed method solves effectively the stereo matching problem and is applicable to various areas.

1. Sun, T. H.(2007).Improving stereo matching quality with scanline-based asynchronous Hopfield neural networks.Journal of the Chinese Institute of Industrial Engineers,24,50-59.
   連結：
2. Ackermann, F.(1984).Digital image correlation-performance and potential application in photogrammetry.Photgrammetric Record,11,429-439.
3. Calitz, M. F.,H. Ruether(1985).Least absolute deviation (LAD) image matching.ISPRS Journal of Photogrammetry and Remote Sensing,52,160-168.
4. Cavalieri, S.,A. D. Stefano,O. Mirabella(1994).Optimal path determination in a graph by Hopfield neural network.Neural Networks,7,387-404.
5. Craievich, D.,B. Barnett,A. C. Bovik(1999).A stereo visual pattern image coding system.Image and Vision Computing,18,21-37.
6. Do, K. H.,Y. S. Kim,T. U. Uam,Y. H. Ha(1998).Iterative relaxational stereo matching based on adaptive support between disparities.Pattern Recognition,31,1049-1059.
7. Foerstner, W.(1982).On the geometric precision of digital correlation.International Archives of Photogrammetry,24,176-189.
8. Giaquinto, N.,M. Savino,S. Taraglio(2002).A CNN-based passive optical range finder for real-time robotic applications.IEEE Transactions on Instrumentation and Measurement,51,314-319.
9. Gong, M.,Y. H. Yang(2005).Fast unambiguous stereo matching using reliability-based dynamic programming.IEEE Transactions on Pattern Analysis and Machine Intelligence,27,998-1003.
10. Grimson, W.(1985).Computational experiments with a feature based stereo algorithm.IEEE Transactions on Pattern Analysis and Machine Intelligence,7,17-34.
11. Gruen, A.(1985).Adaptive least square correlation: a powerful image matching technique.ISPRS Journal of Photogrammetry, Remote Sensing and Cartography,14,175-187.
12. Gruen, A.,P. Agouris(1994).Linear extraction by least squares template matching constrained by internal forces.Proceedings of ISPRS Commission III Symposium on Spatial Information from Digital Photogrammetry and Computer Vision,Munich, German:
13. Hertz, J.,A. Krogh,R. G. Palmer(1991).Introduction to the Theory of Neural Computation.MA:Addison-Wesley.
14. Hopfield, J.(1982).Neural networks and physical systems with emergent collective computational abilities.Proceedings of the National Academy of Science of the United States of America,79,2554-2558.
15. Hopfield, J.(1982).Neurons with graded response have collective computational properties like those of two-state neurons.Proceedings of the National Academy of Science of the United States of America,81,3088-3092.
16. Hopfield, J.,D. W. Tank(1985).Neural computation of decisions in optimization problems.Biological Cybernetics,52,141-152.
17. Hu, J. E.,P. Siy(1993).An ordering-oriented Hopfield network and its application in stereo vision.The Smithsonian/NASA Astrophysics Data System,1965,556-567.
18. Huang, M. S.,T. S. Lew,K. Wong(1994).Learning and feature selection in stereo matching.IEEE Transactions on Pattern Analysis and Machine Intelligence,16,869-881.
19. Lee, J. J.,J. C. Shim,U. H. Ha(1994).Stereo correspondence using the Hopfield neural network of a new energy function.Pattern Recognition,27,1513-1522.
20. Lee, S. H.,J. J. Leou(1994).A dynamic programming approach to line segment matching in stereo vision.Pattern Recognition,27,961-986.
21. Medioni, G.,R. Nevatia(1985).Segment based stereo matching.Computer Vision, Graphics, and Image Processing,31,2-18.
22. Moravec, H.(1981).Robot Rover Visual Navigation.MI:UMI Research.
23. Mortara, M. S.,M. Spagnuolo(2001).Similarity measures for bending polygonal shapes.Computer & Graphics,35,13-27.
24. Mousavi, M. S.,R. J. Schalkoff(1990).A parallel distributed algorithm for feature extraction and disparity analysis of computer images.Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition,San Francisco, USA:
25. Mousavi, M. S.,R. J. Schalkoff(1990).Stereo vision: a neural network application to constraint satisfaction problem.Proceedings of the SPIE Intelligent Robots and Computer Vision IX: Neural, Biological, and 3-D Methods,1382,228-239.
26. Mousavi, M. S.,R. J. Schalkoff(1990).A parallel distributed algorithm for feature extraction and disparity analysis of computer images.Proceedings of the IEEE Computer Society Conference on Computer Vision on Pattern Recognition,Dallas, USA:
27. Nasrabadi, N. M.,C. Y. Choo(1992).Hopfield network for stereo vision correspondence.IEEE Transactions on Neural Networks,3,5-13.
28. Nasrabadi, N. M.,W. Li(1991).Object recognition by a Hopfield neural network.IEEE Transactions on Systems, Man, and Cybernetics,21,1523-1535.
29. Ohta, Y.,T. Kanade(1985).Stereo by intraand-inter-scanline search using dynamic programming.IEEE Transactions on Pattern Analysis and Machine Intelligence,7,139-154.
30. Pajares, G.,J. M. Cruz,J. Aranda(1997).Relaxation by Hopfield network in stereo image matching.Pattern Recognition,31,561-574.
31. Parvin, B.,G. Medioni(1991).A layered network for the correspondence of 3D objects.Proceedings of the IEEE International Conference on Robotics and Automation,Sacramento, USA:
32. Ruicheck, Y.,J. G. Postaire(1995).A neural implementation for high speed processing in linear stereo vision.Proceedings of the IEEE International Conference on Systems, Man and Cybernetics,Vancouver, Canada:
33. Sarkar, I.,M. Bansal(2007).A wavelet-based multiresolution approach to solve the stereo correspondence problem using mutual information.IEEE Transaction on Systems, Man, and Cybernetics-Part B: Cybernetics,37,1009-1014.
34. Shirai, Y.(1987).Three-dimensional Computer Vision.NY:Springer-Verlag.
35. Sun, J.,N. N. Zheng,H. Y. Shum(2003).Stereo matching using belief progapation.IEEE Transactions on Pattern Analysis and Machine Intelligence,25,787-800.
36. Tank, D.,J. Hopfield(1986).Simple 'neural' optimization networks: an A/D converter, signal decision circuit, and a linear programming circuit.IEEE Transactions on Circuits and Systems,33,533-541.
37. Tien, F. C.,C. A. Chang(1999).Neural network for precise 3D measurement in stereo vision system.International Journal of Production Research,37,1935-1948.
38. Tien, F. C.,C. Y. Tsai(2004).Scanline-based stereo matching by genetic algorithm.International Journal of Production Research,42,1083-1106.
39. Tsin, Y.,S. B. Kang,R. Szeliski(2006).Stereo matching with linear superposition of layers.IEEE Transactions on Pattern Analysis and Machine Intelligence,28,290-301.
40. Wolfe, W. J.(1991).A parallel approach to simultaneously solving the correspondence problem and the pose estimation problem.Proceeding of the SPIE Conference on Mobile Robots VI,Boston:
41. Wolfe, W. J.,M. Magee(1990).Fusion of multiple views of multiple reference points using a parallel distributed processing approach.Proceeding of the SPIE Conference on Sensor Fusion III,Boston:
42. Zhang, L.(2006).Fast stereo matching algorithm for intermediate view reconstruction of stereoscopic television images.IEEE Transaction on Circuits and Systems for Video Technology,16,1259-1270.
43. Zurada, J. M.(1992).Introduction to Artificial Neural Systems.NY:West Publishing Company.