基於多重解析度環場視訊的沉浸式遙現之研究

A Study on Immersive Telepresence with Multi-Resolution Panoramic Video

10.6342/NTU201702257

黃昱琪

多重解析度 ； 遙現 ； 串流影片 ； 網路延遲 ； 頭戴式顯示器 ； multi-resolution ； telepresence ； streaming video ； network latency ； head-mounted display

國立臺灣大學資訊工程學系學位論文

2017年

碩士

洪一平

英文

在最近虛擬實境科技的發展下，沉浸式的遙現逐漸變得熱門。沉浸 式遙現系統為了要讓使用者能自由控制視角，需要提供環場的影像。 而由於環場影像需錄下全方位的影像資訊，必須得使用到更高的解析 度來錄製。這樣會讓使用者必須要使用高效能的裝置才能夠觀看，並 且系統必須要處理龐大的網路流量。在這篇論文中，我們提出使用多 重解析度的方法去解決此問題。我們觀察到在沉浸式的遙現，使用者 在同一時間只會看到部分的環場影像。所以，我們只需要提供使用者 正在觀看區域的高解析度影像，如此一來能節省許多傳輸不必要資訊 的流量。然而，由於存在網路延遲，使用者若要轉換視角，系統無法 馬上更新。因此，我們也把低解析度的環場影像一起傳輸，以此讓使 用者還能看到環場的影像。我們並評估了在使用多重解析度環場的情 況下，網路延遲對系統易用性的影響。結果顯示網路延遲在 0.5 秒內， 對易用性的並沒有很大的影響。在現今的網路與硬體條件下，要控制 到 0.5 秒以下的延遲並不是很困難。由此結果我們可以得出結論:多 重解析度是個沉浸式遙現的可行解決方案。

As the development of VR technology in recent years, the immersive telepresence system has becoming more and more popular. The immersive telepresence system often need to provide whole surrounding omnidirectional view to allow users to freely control their viewing direction. With whole sur- rounding information introduced, higher resolution video is needed. How- ever, this will require users to have powerful devices in order to play the video. Also, the system need to handle massive network bandwidth. In this thesis, we proposed using multi-resolution method to address this problem. Observ- ing that only portion of the panorama will be needed at a time, we only need provide high resolution video in user’s visible portion. In this way, we can saved a lot of bandwidth. However, the view cannot be updated instanstly due to the network latency. To overcome the issue, we also provide a low resolu- tion panoramic video to compensate the latency. In our evaluation, we futher investigated the effect of latency on usability of the multi-resolution method. Our result shows that latency have small effect on usability under 0.5 seconds. For the current hardware and network standard, it is not hard to control the latency under 0.5 seconds. We therefore conclude that the multi-resolution method is a feasible solution for immersive telepresence system.

1. [2] J. Y. Chen, E. C. Haas, and M. J. Barnes. Human performance issues and user interface design for teleoperated robots. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 37(6):1231–1245, 2007.
   連結：
2. [3] K.-W. Chen, C.-W. Lin, M. Y. Chen, and Y.-P. Hung. e-fovea: a multi-resolution approach with steerable focus to large-scale and high-resolution monitoring. In Proceedings of the 18th ACM international conference on Multimedia, pages 311–320. ACM, 2010.
   連結：
3. [5] J. Dichgans and T. Brandt. Optokinetic motion sickness and pseudo-coriolis effects induced by moving visual stimuli. Acta oto-laryngologica, 76(1-6):339–348, 1973.
   連結：
4. [6] R. S. Kennedy, N. E. Lane, K. S. Berbaum, and M. G. Lilienthal. Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness. The international journal of aviation psychology, 3(3):203–220, 1993.
   連結：
5. [7] N. Q. M. Khiem, G. Ravindra, and W. T. Ooi. Towards understanding user tolerance to network latency in zoomable video streaming. In Proceedings of the 19th ACM international conference on Multimedia, pages 977–980. ACM, 2011.
   連結：
6. [8] K. Kim, M. Z. Rosenthal, D. Zielinski, and R. Brady. Comparison of desktop, head mounted display, and six wall fully immersive systems using a stressful task. In Virtual Reality Short Papers and Posters (VRW), 2012 IEEE, pages 143–144. IEEE, 2012.
   連結：
7. [10] J.-W. Lin, H. B.-L. Duh, D. E. Parker, H. Abi-Rached, and T. A. Furness. Effects of field of view on presence, enjoyment, memory, and simulator sickness in a virtual environment. In Virtual Reality, 2002. Proceedings. IEEE, pages 164–171. IEEE, 2002.
   連結：
8. [11] H. Martins and R. Ventura. Immersive 3-d teleoperation of a search and rescue robot using a head-mounted display. In Emerging Technologies & Factory Automation, 2009. ETFA 2009. IEEE Conference on, pages 1–8. IEEE, 2009.
   連結：
9. [14] O. A. Niamut, E. Thomas, L. D’Acunto, C. Concolato, F. Denoual, and S. Y. Lim. Mpeg dash srd: Spatial relationship description. In Proceedings of the 7th Interna- tional Conference on Multimedia Systems, page 5. ACM, 2016.
   連結：
10. [15] E. Nystad and A. Sebok. A comparison of two presence measures based on experimental results. In Proceedings of the seventh international workshop on presence, pages 266–273, 2004.
    連結：
11. [17] Y. Ren and H. Fuchs. Faster feedback for remote scene viewing with pan-tilt stereo camera. In Virtual Reality (VR), 2016 IEEE, pages 273–274. IEEE, 2016.
    連結：
12. [18] T. B. Sheridan. Musings on telepresence and virtual presence. Presence: Teleoperators & Virtual Environments, 1(1):120–126, 1992.
    連結：
13. [19] M. Slater, M. Usoh, and A. Steed. Depth of presence in virtual environments. Pres- ence: Teleoperators & Virtual Environments, 3(2):130–144, 1994.
    連結：
14. [20] T. Stockhammer. Dynamic adaptive streaming over http–: standards and design principles. In Proceedings of the second annual ACM conference on Multimedia systems, pages 133–144. ACM, 2011.
    連結：
15. [21] B. G. Witmer and M. J. Singer. Measuring presence in virtual environments: A presence questionnaire. Presence: Teleoperators and virtual environments, 7(3):225– 240, 1998.
    連結：
16. [1] Insta 360. https://www.insta360.com. Accessed: 2017-07-10.
17. [4] O. Cogal, A. Akin, K. Seyid, V. Popovic, A. Schmid, B. Ott, P. Wellig, and Y. Leblebici. A new omni-directional multi-camera system for high resolution surveillance. SPIE Sensing Technology+ Applications, pages 91200N–91200N, 2014.
18. [9] J. C. Lane, C. R. Carignan, B. R. Sullivan, D. L. Akin, T. Hunt, and R. Cohen. Effects of time delay on telerobotic control of neutral buoyancy vehicles. In Robotics and Automation, 2002. Proceedings. ICRA’02. IEEE International Conference on, volume 3, pages 2874–2879. IEEE, 2002.
19. [12] M. Meehan, S. Razzaque, M. C. Whitton, and F. P. Brooks. Effect of latency on presence in stressful virtual environments. In virtual reality, 2003. Proceedings. IEEE, pages 141–148. IEEE, 2003.
20. [13] T.Ni, D.A.Bowman, and J.Chen. Increased display size and resolution improve task performance in information-rich virtual environments. In Proceedings of Graphics Interface 2006, pages 139–146. Canadian Information Processing Society, 2006.
21. [16] N. Quang Minh Khiem, G. Ravindra, A. Carlier, and W. T. Ooi. Supporting zoomable video streams with dynamic region-of-interest cropping. In Proceedings of the first annual ACM SIGMM conference on Multimedia systems, pages 259–270. ACM, 2010.