適用於自動影像去背之基於深度及多重視角產生影像三元圖演算法

Multi-View Depth-Based Trimap Generation for Automatic Natural Image Matting

黃俊穎

影像去背 ； 多重視角 ； 影像三元圖 ； image matting ； multi-view ； trimap

清華大學電機工程學系所學位論文

2016年

碩士

黃朝宗

英文

影像去背為一近期很受歡迎的影像處理方法，他能將一張照片的前景取出，並保有邊緣(如頭髮等)的半透明成分。現有的影像去背演算法幾乎都是半自動的，他們需要額外的使用者介入才能真正計算出所有的前景透明度。影像三元圖是其中最常見的使用者介入表達方法其中最常見的使用者介入表達方法，使用者事先將影像中無透明度的前景以及背景位置標示成白色和黑色，剩餘有透明度的前景部分則標為灰色。影像三元圖的優劣會直接影響影像去背的品質。然而，產生一張優質的影像三元圖是很消耗時間的一件事情，所以我們提出一個能夠自動產生影像三元圖的方法，讓整個影像去背變成一個全自動的流程。 我們利用Lytro光場相機所提供的多重視角影像以及深度資訊來建構我們的自動產生影像三元圖演算法，這些資訊能夠幫助我們自動辨識出哪些區域應該要被標記為未知區域(灰色)。在利用這些資訊之後我們會產生一張初始三元圖，再來我們還會根據多重視角的優勢制訂一種簡單有效率的前景透明度估計法，藉此將我們的三元圖修飾的更加準確，並產生最終的影像三元圖。 我們不僅成功的將整套影像去背流程變為全自動化，還會利用主觀以及客觀的比較實驗來證明我們的影像三元圖能夠同時增進影像去背演算法的準確度以及運算效率。本篇論文以closed-form matting演算法為例，該演算法的未知區域大小正比於運算時間，而我們演算法產生出來的影像三元圖能夠讓該演算法有更好以及更有效率的影像去背結果。

Image matting is the problem of extracting foreground components from an image. Among all matting algorithms, additional user input is required during matting process since it is inherently an under-constrained problem. A trimap is a general and popular user input that segments the image into three parts: foreground, background, and unknown region. The accuracy of a trimap directly affects the computation time and matting quality for alpha matting algorithms. However, a good trimap is usually drawn manually in literature, which is a tedious and time-consuming process. We aim to design an algorithm to generate a trimap that requires only simple user input but delivers good results for the followed alpha matting algorithm. With the help of light field capturing, we use multi-view and depth information to design an automatic trimap generation. Based on them, we segment the original image and generate the corresponding initial trimap. We also design a background sampling-based algorithm which is used to refine the initial trimap to generate the final trimap. We will show that our trimap can increase not only estimation accuracy but also computation efficiency for image matting. Both subjective and objective experiments will be given to demonstrate these advantages. Take closed-form matting for example, we will prove the number of pixels in unknown region is proportional to computation time, so the more foreground and background pixels we specify in our trimap, the less computation time needed in matting process.

1. [5] A. Levin, D. Lischinski, and Y. Weiss, "A Closed Form Solution to Natural ImageMatting," IEEE Trans Pattern Anal Mach Intell, vol. 30, pp. 228-242, 2008.
   連結：
2. [6] A. Levin, A. Rav-Acha, and D. Lischins, "Spectral Matting," IEEE Trans Pattern Anal Mach Intell, pp. 1699 - 1712, 2008.
   連結：
3. [8] X. Bai and G. Sapiro, "Geodesic Matting: A Framework for Fast Interactive Image and Video Segmentation and Matting," International Journal of Computer Vision, vol. 82, pp. 113-132, 2008.
   連結：
4. [9] J. Wang and M. F. Cohen, "Optimized Color Sampling for Robust Matting," in IEEE Conference on Computer Vision and Pattern Recognition, 2007, pp. 1-8.
   連結：
5. [11] J. Wang and M. F. Cohen. (2007, Image and Video Matting- A Survey. 97-175
   連結：
6. [12] N. Joshi, W. Matusik, and S. Avidan, "Natural Video Matting using Camera Arrays," ACM Transactions on Graphics (TOG) vol. 25, pp. 779-786 2006.
   連結：
7. [13] About LYTRO. Available: https://www.lytro.com/about
   連結：
8. [14] S. Singh, A. S. Jalal, and C. Bhatanagar, "Automatic Trimap and Alpha-Matte Generation For Digital Image Matting," presented at the Sixth International Conference on Contemporary Computing 2013.
   連結：
9. [15] W. H. Cheng, "Algorithm and Architecture Study on Disparity Map Synthesis for Free Viewpoint Syhthesis of Sparse Light Field," Master Degree, Department of Electrical Engineering, National Tsing Hua University, 2016.
   連結：
10. [1] A. R. Smith and J. F. Blinn, "Blue screen matting," in 23rd annual conference on Computer graphics and interactive techniques, 1996.
11. [2] Y. Y. Chuang, B. Curless, D. H. Salesin, and R. Szeliski, "A Bayesian Approach to Digital Matting," in 2001 IEEE Computer Society Conference, 2001.
12. [3] J. Sun, J. Jia, C.-K. Tang, and H.-Y. Shum, "Poisson Matting," ACM SIGGRAPH, vol. 23, pp. 315-321 2004.
13. [4] L. Grady, T. Schiwietz, S. Aharon, and R. Westermann, "Random walks for interactive alpha-matting," in Proceedings of VIIP, 2005, pp. 423-429.
14. [7] Y. Guan, W. Chen, X. Liang, Z. a. Ding, and Q. Peng, "Easy Matting - A Stroke Based Approach for Continuous Image Matting," EUROGRAPHICS, vol. 25, pp. 567-576, 2006.
15. [10] Y. Zheng and C. Kambhamettu, "Learning based digital matting," presented at the IEEE 12th International Conference on Computer Vision, 2010.