自監督式深度學習影像匹配應用於福衛光學衛星影像幾何校正

Self-supervised Deep-learning-based Image Matching for FORMOSAT Optical Satellite Image Orthorectification

10.6574/JPRS.202306_28(2).0001

吳菉(Lu Wu)；張雅筑(Ya-Chu Chang)；林柏毅(Bo-Yi Lin)；林昭宏(Chao-Hung Lin)；曾義星(Yi-Hsing Tseng)；張立雨(Li-Yu Chang)；張莉雪(Li-Hsueh Chang)；李彥玲(Yen-Ling Lee)

幾何校正 ； 有理函數模型 ； 深度學習 ； 基於特徵的影像匹配 ； 光學衛星影像 ； Optical Satellite Image ； Image Orthorectification ； Rational Function Model ； Deep Learning ； Feature-Based Image Matching

航測及遙測學刊

28卷2期（2023 / 06 / 01）

63 - 81

繁體中文;英文

標準幾何校正流程在獲取控制點上花費大量人力及時間，為使衛星影像呈現精確的幾何成像，且提升獲取衛星影像之效率，本研究提出一新穎的自動化衛星影像幾何校正流程。藉由自監督深度學習影像匹配演算法及影像匹配策略，於衛星影像中自動化獲取更多的顯著特徵作為影像控制點，使得衛星影像幾何校正流程更穩健且便捷。實驗結果表明，自動化幾何校正流程不僅具穩定性且具適應性，幾何校正結果在福衛五號2米空間解析度下誤差約為2至4像元。

The standard orthorectification process takes a lot of manpower and time to obtain control points. To correctly represent the image geometry on satellite images and improve the efficiency of satellite image orthorectification, a novel method for automatic satellite image orthorectification is proposed. In this study, a robust satellite image matching process is processed to obtain image control points, which adopted. Different from traditional labor-intensive methods, a novel image matching method is adopted to find image control points both on target images and an orthorectified reference image, which is adopted self-supervised deep learning image matching algorithm. This strategy makes the ortho-rectification process become automatic, robust, and attempts to distinguish more salient features than traditional methods in satellite images. The experimental results show that the automatic orthorectification process is not only stable but also adaptable. The quantity assessment is performed using root mean square error, and the accuracy of satellite image orthorectification result is 2 to 4 pixels under the 2-meter spatial resolution of FORMOSAT-5 images.

1. Bay, H.,Ess, A.,Tuytelaars, T.,Van Gool, L.(2008).Speeded-up robust features (SURF).Computer Vision and Image Understanding,110(3),346-359.
2. Cao, J.,Fu, J.(2018).Estimation of rational polynomial coefficients based on singular value decomposition.Journal of Applied Remote Sensing,12(4),044003.
3. Chen, L.,Rottensteiner, F.,Heipke, C.(2021).Feature detection and description for image matching: From hand-crafted design to deep learning.Geo-spatial Information Science,24(1),58-74.
4. Chen, L.C.,Teo, T.A.,Liu, C.L.(2006).The geometrical comparisons of RSM and RFM for FORMOSAT-2 satellite images.Photogrammetric Engineering and Remote Sensing,72(5),573-579.
5. DeTone, D.,Malisiewicz, T.,Rabinovich, A.(2017).,未出版
6. DeTone, D.,Malisiewicz, T.,Rabinovich, A.(2018).Superpoint: Self-supervised interest point detection and description.Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW),Salt Lake City, UT, USA:
7. Kim, T.,Shin, D.,Lee, Y.R.(2001).Development of a robust algorithm for transformation of a 3D object point onto a 2D image point for linear pushbroom imagery.Photogrammetric Engineering and Remote Sensing,67(4),449-452.
8. Kratky, V.(1989).Rigorous photogrammetric processing of SPOT images at CCM Canada.ISPRS Journal of Photogrammetry and Remote Sensing,44(2),53-71.
9. Lebedev, M.A.,Vizilter, Y.V.,Vygolov, O.V.,Knyaz, V.A.,Rubis, A.Y.(2018).Change detecion in remote sensing images using conditional adversarial networks.The International Archives of the Photogrammetry, Remote Sensing & Spatial Information Sciences,42(2),565-571.
10. Lin, T.Y.,Maire, M.,Belongie, S.,Hays, J.,Perona, P.,Ramanan, D.,Dollár, P.,Zitnick, C.L.(2014).Microsoft coco: Common objects in context.Proceedings of the European Conference on Computer Vision,Zurich, Switzerland:
11. Lowe, D.G.(2004).Distinctive image features from scale-invariant keypoints.International Journal of Computer Vision,60(2),91-110.
12. Muja, M.,Lowe, D.G.(2009).Fast approximate nearest neighbors with automatic algorithm configuration.Proceedings of the Fourth International Conference on Computer Vision Theory and Applications
13. O'NEILL, M.(1988).The generation of epipolar synthetic stereo mates for SPOT images using a DEM.International Archives of Photogrammetry and Remote Sensing,27,587-598.
14. Pizer, S.M.,Johnston, R.E.,Ericksen, J.P.,Yankaskas, B.C.,Muller, K.E.(1990).Contrast-limited adaptive histogram equalization: Speed and effectiveness.Proceedings of the First Conference on Visualization in Biomedical Computing,Atlanta, GA, USA:
15. Poli, D.,Toutin, T.(2012).Review of developments in geometric modelling for high resolution satellite pushbroom sensors.The Photogrammetric Record,27(137),58-73.
16. Rosten, E.,Drummond, T.(2006).Machine learning for high-speed corner detection.Proceedings of the European Conference on Computer Vision (ECCV),Berlin, Heidelberg:
17. Sarlin, P.E.,DeTone, D.,Malisiewicz, T.,Rabinovich, A.(2020).Superglue: Learning feature matching with graph neural networks.Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,Seattle, WA, USA:
18. Tong, X.,Liu, S.,Weng, Q.(2010).Bias-corrected rational polynomial coefficients for high accuracy geo-positioning of QuickBird stereo imagery.ISPRS Journal of Photogrammetry and Remote Sensing,65(2),218-226.
19. Toutin, T.(2004).Geometric processing of remote sensing images: Models, algorithms and methods.International Journal of Remote Sensing,25(10),1893-1924.
20. Wiesel, J.(1985).Digital image processing for orthophoto generation.Photogrammetria,40(2),69-76.
21. Yildirim, I.,Demirtas, F.,Gulmez, B.,Leloglu, U.M.,Yaman, M.,Guneyi, E.T.(2019).Comparison of image matching algorithms on satellite images taken in different seasons.Proceedings of the Türkiye Ulusal Fotogrametri ve Uzaktan Algılama Birliği Teknik Sempozyumu (TUFUAB’2019),Aksaray, Türkiye:
22. Zhou, Q.,Sattler, T.,Leal-Taixe, L.(2021).Patch2pix: Epipolar-guided pixel-level correspondences.Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR),Nashville, TN, USA:
23. 林義乾, Y.C.(2006)。Taiwan, ROC，國立臺灣大學土木工程學系=National Taiwan University。
24. 張智安, T.A.,陳良健, L.C.(2003)。EROS A 衛星影像幾何改正之研究。航測及遙測學刊，8(3)，73-94。
25. 陳信安, H.A.(2007)。Taiwan, ROC，國立臺灣大學地理環境資源學系=National Taiwan University。
26. 陳俊愷, C.K.(2011)。Taiwan, ROC，國立臺灣師範大學地理學系=National Taiwan Normal University。