基於自監督學習之超解析度成像法應用於Sentinel-2衛星影像

SELF-SUPERVISED SUPER-RESOLUTION ON SENTINEL-2 IMAGERY

10.6652/JoCICHE.202205_34(3).0007

虞景翔(Ching-Hsiang Yu)；謝孟財(Mon-Chai Hsieh)；任玄(Hsuan Ren)

衛星影像 ； 卷積神經網路 ； 超解析度成像 ； 自監督學習 ； satellite imagery ； convolutional neural network ； super-resolution ； self-supervised

中國土木水利工程學刊

34卷3期（2022 / 05 / 01）

243 - 248

繁體中文

近年卷積神經網路廣泛運用於超解析度成像法，這類監督式學習法能利用外部影像集訓練模型以提升特定特徵目標如人臉或建物的影像解析度，訓練過程耗費大量計算資源。本研究測試Assaf Shocher所開發Zero-shot Super-resolution（ZSSR）方法，應用提升Sentinel-2衛星影像之空間解析度。ZSSR屬於自監督學習方法，不需大量影像集的訓練，只針對測試影像本身內部結構特徵進行學習。實驗結果相較於傳統雙三次插值在峰值訊噪比（PSNR）及結構相似性（SSIM）能提升4.67%及3.35%，與耗費訓練資源的監督式學習方法成效相當。成果也顯示遙測影像中具重複性的內部結構特徵，適合應用自監督學習超解析度成像法提高空間解析度。

Convolutional neural networks have been adopted in super-resolution algorithm inrecent years. These supervised learning methods can train the model through external image sets to improve the image resolution of specific feature such as faces or buildings. Because they need large amount of training images, it usually consumes lots of computation cost. This study implements the Zero-shot Super-resolution (ZSSR) method developed by Assaf Shocher and applied to improve the spatial resolution of Sentinel-2 images. ZSSR is a self-supervised learning method that does not need the pre-training process with image data set. It only needs to learn the internal structural features of the test image itself. The experimental results show that ZSSR can improve peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) by 4.67% and 3.35% compared with bicubic interpolation, which are comparable to supervised learning methods that consume training resources. The results also show, that the repeated internal structural features in remote sensing images are suitable for self-supervised learning super-resolution algorithms.

1. Chao, D.,Loy, C. C.,He, K.,Tang, X.(2016).Image super-resolution using deep convolutional networks.IEEE Transactions on Pattern Analysis and Machine Intelligence,38(2),295-307.
2. Demir, I.,Koperski, K.,Lindenbaum, D.,Pang, G.,Huang, J.,Basu, S.,Hughes, F.,Tuia, D.,Raskar, R.(2018).DeepGlobe 2018: A challenge to parse the earth through satellite images.IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW),UT, USA:
3. He, K.,Zhang, X.,Ren, S.,Sun, J.(2016).Deep residual learning for image recognition.IEEE Conference on Computer Vision and Pattern Recognition (CVPR),NV, USA:
4. Lecun, Y.,Bottou, L.,Bengio, Y.,Haffner, P.(1998).Gradient-based learning applied to document recognition.Proceedings of the IEEE,86(11),2278-2324.
5. Ledig, C.,Theis, L.,Huszar, F.,Caballero, J.,Cunningham, A.,Acosta, A.,Aitken, A.,Tejani, A.,Totz, J.,Wang, Z.,Shi, W.(2017).Photo-realistic single image super-resolution using a generative adversarial network.IEEE Conference on Computer Vision and Pattern Recognition (CVPR),HI, USA:
6. Lim, B.,Son, S.,Kim, H.,Nah, S.,Lee, K. M.(2017).Enhanced deep residual networks for single image super-resolution.IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW),HI, USA:
7. Shi, Y.,Li, B.,Wang, B.,Qi, Z.,Liu, Ji(2019).Unsupervised single-image super-resolution with multigram loss.Electronics,8(8),833.
8. Shocher, A.,Cohen, N.,Irani, M.(2018).Zero-shot super-resolution using deep internal learning.IEEE/CVF Conference on Computer Vision and Pattern Recognition,UT, USA:
9. Zontak, M.,Irani, M.(2011).Internal statistics of a single natural image.IEEE Conference on Computer Vision and Pattern Recognition (CVPR),CO, USA:
10. 陳昭偉(2008)。新竹，中華大學資訊工程學系研究所。
11. 黃吉宏,黃明鴻,吳宗達,陳明麗(2001)。微型光學系統於微微衛星之應用。科儀新知，23(2)，27-43。