配戴頭戴顯示器觀看360度全景影片愈趨流行,然而在這領域理解 使用者體驗品質(QoE)是一項很大的挑戰,因為使用者體驗被諸多因素 所影響。本論文致力於提出使用者體驗品質模型以預測配戴頭戴顯示 器觀看360度全景影片的使用者體驗。為了達到此目標,我們實作了 一360度全景影片播放器,其支援多種不同的投影方式,此影片播放 器可支援4K解析度和每秒30幀數的360度全景影片。為了探索會影響 使用者體驗的顯著因素,我們設計一使用者實驗,並考慮以下因素: (一)投影方式、(二)壓縮量化參數及(三)影片特質,我們分析 其因素與使用者體驗之間的相關程度。我們更根據以上實驗結果提出 了多個使用者體驗品質模型,以預測使用者的360度全景影片觀看體 驗,我們也針對不同使用情境推薦了其適合的模型。

Watching 360° videos with a Head-Mounted Display (HMD) is getting popular. However, understanding Quality-of-Experience (QoE) of 360° videos with HMDs is quite challenging because it may be affected by too many fac- tors. This thesis strives to develop models to predict the QoE levels of watch- ing 360° videos with HMDs. To achieve that, first, we implement a 360° video player supporting diverse projection schemes. The 360° video player can support 4K resolution videos at 30 frame-per-second. Then, we conduct a user study to explore the implications of different factors on the QoE of 360° videos. The subjective experiments are designed to exercise: (i) pro- jection schemes, (ii) encoding Quantization Parameters (QPs), and (iii) video genres. We analyze the correlation of the three factors and objective quality metrics with ground-truth QoE levels. Based on the subjective and objec- tive assessments, we construct multiple QoE models to predict the QoE of 360° videos. We also offer our recommendation on model selections under different circumstances.

口試委員會審定書 i
Acknowledgments ii
致謝 iii
中文摘要 iv
Abstract v
1 Introduction 1
1.1 Contributions ................................ 2
1.2 Thesis Organization............................. 3
2 Background 4
2.1 360° Videos................................. 4
2.2 Virtual Reality................................ 6
2.3 Viewing Experience............................. 6
3 360° Video Player for Diverse Projection Schemes 9
3.1 Existing Open-source 360°Video Players ................. 9
3.2 OSVR-based 360° Video Player ...................... 10
3.3 Diverse Projection Schemes: Design and Implementation . . . . . . . . . 12
3.4 Performance Measurements......................... 12
4 User Study 15
4.1 Setup .................................... 15
4.2 Procedure.................................. 18
4.3 Subjective Analysis Results......................... 19
4.4 Objective Analysis Results ......................... 24
5 Quality-of-Experience Modeling 26
5.1 Potential Factors .............................. 27
5.2 Virtual Reality 360° Video QoE Model................... 27
5.3 Validation.................................. 29
5.4 Evaluation.................................. 30
6 Related Work 32
6.1 Players.................................... 32
6.2 User Study ................................. 33
6.3 Modeling .................................. 34
7 Discussions 36
7.1 Major Findings ............................... 36
7.2 Application Scenarios............................ 37
7.3 Limitations and Future Directions...................... 37
8 Conclusion 39
Bibliography 40

[1] M. Alreshoodi and J. Woods. Survey on QoE/QoS correlation models for multime- dia services. International Journal of Distributed and Parallel Systems, 4(3), May 2013.
[2] L. Anegekuh, L. Sun, and E. Ifeachor. Encoding and video content based HEVC video quality prediction. Multimedia Tools and Applications, 74(11):3715–3738, June 2015.
[3] S. Barakovic ́ and L. Skorin-Kapov. Survey and challenges of QoE management issues in wireless networks. Computer Networks and Communications, December 2012.
[4] K. Calagari, M. Elgharib, S. Shirmohammadi, and M. Hefeeda. Sports VR content generation from regular camera feeds. In Proc. of ACM International Conference on Multimedia (MM’17), pages 699–707, Mountain View, USA, October 2017.
[5] S. Channappayya, A. Bovik, C. Caramanis, and R. Heath. SSIM-optimal linear image restoration. In Proc. of IEEE International Conference on Acoustics, Speech and Signal Processing, pages 765–768, Las Vegas, USA, March 2008.
[6] G. Charness, U. Gneezy, and M. Kuhn. Experimental methods: Between-subject and within-subject design. Economic Behavior & Organization, 81(1):1–8, January 2012.
[7] G. Charness, U. Gneezy, and M. A. Kuhn. Experimental methods: Between-subject and within-subject design. Economic Behavior & Organization, 81(1):1–8, January 2012.
[8] L. Christensen, B. Johnson, and L. Turner. Experimental research design. In Re- search methods, design, and analysis, chapter 8, pages 217–248. Allyn & Bacon Boston, MA, 2011.
[9] X. Corbillon, F. De Simone, and G. Simon. 360-degree video head movement dataset. In Proc. of the 8th ACM on Multimedia Systems Conference (MMSys’17), pages 199–204, Taipei, Taiwan, June 2017.
[10] X. Corbillon, G. Simon, A. Devlic, and J. Chakareski. Viewport-adaptive navigable 360-degree video delivery. In Proc. of IEEE International Conference on Commu- nications (ICC’17), pages 1–7, Paris, France, May 2017.
[11] I. D. Curcio, H. Toukomaa, and D. Naik. Bandwidth reduction of omnidirectional viewport-dependent video streaming via subjective quality assessment. In Proc. of the 2Nd International Workshop on Multimedia Alternate Realities (AltMM’17), pages 9–14, Mountain View, California, USA, October 2017.
[12] ExoPlayer : An extensible media player for android, July 2018. https:// github.com/google/ExoPlayer.
[13] FFmpeg web page, July 2018. http://www.ffmpeg.org.
[14] C. Fu, L. Wan, T. Wong, and C. Leung. The rhombic dodecahedron map: An ef- ficient scheme for encoding panoramic video. IEEE Transactions on Multimedia, 11(4):634–644, June 2009.
[15] Global virtual reality market (hardware and software) and forecast to 2020, June 2018. https://goo.gl/vtFXVK.
[16] Google AR and VR Blog web page, July 2018.
https://blog.google/products/google-ar-vr/ bringing-pixels-front-and-center-vr-video/.
[17] Google Cardboard, July 2018. https://vr.google.com/cardboard/.
[18] GPAC: Multimedia open source project, October 2018. https://gpac.wp.
imt.fr/.
[19] M.Graf,C.Timmerer,andC.Mueller.Towardsbandwidthefficientadaptivestream- ing of omnidirectional video over HTTP: Design, implementation, and evaluation. In Proc. of the 8th ACM on Multimedia Systems Conference (MMSys’17), pages 261–271, Taipei, Taiwan, June 2017.
[20] Microsoft HoloLens, October 2018. https://www.microsoft.com/ en-us/hololens.
[21] A. Hore and D. Ziou. Image quality metrics: PSNR vs. SSIM. In Pattern Recogni- tion (ICPR), 2010 20th International Conference on, pages 2366–2369. IEEE, 2010.
[22] T. Hoßfeld, L. Skorin-Kapov, Y. Haddad, P. Pocta, V. A. Siris, A. Zgank, and H. Melvin. Can context monitoring improve QoE? a case study of video flash crowds in the internet of services. In Proc. of IEEE International Symposium on Integrated Network Management (IM’15), pages 1274–1277, Ottawa, ON, Canada, May 2015.
[23] C. Hsu, A. Chen, C. Hsu, C. Huang, C. Lei, and K. Chen. Is foveated rendering perceivable in virtual reality? exploring the efficiency and consistency of quality assessment methods. In Proceedings of the 2017 ACM on Multimedia Conference. ACM, 2017.
[24] Q. Huynh-Thu and M. Ghanbari. Scope of validity of PSNR in image/video quality assessment. Electronics Letters, 44(13):800–801, June 2008.
[25] ITU-R. BT.500-13: Methodology for the subjective assessment of the quality of television pictures. Technical report, January 2012.
[26] ITU-T. J.247 Objective perceptual multimedia video quality measurement in the presence of a full reference. Technical report, August 2008.
[27] ITU-T. P.910 Subjective video quality assessment methods for multimedia applica- tions. Technical report, April 2008.
[28] ITU-T. P.800.1 Mean opinion score (MOS) terminology. Technical report, August 2016.
[29] Vocabulary for performance and quality of service. Standard, ITU Telecommunica- tion Standardization Sector, 2016.
[30] W. Jiang and J. Gu. Video stitching with spatial-temporal content-preserving warp- ing. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR’15), pages 42–48, Boston, USA, June 2015.
[31] Joint video exploration team, ”360Lib”, September 2017. https://jvet.hhi. fraunhofer.de/svn/svn_360Lib/tags/360Lib-2.0.1/.
[32] AHG8: Suggested testing procedure for 360-degree video. Technical report, Joint Video Exploration Team of ITU Telecommunication Standardization Sector, Octo- ber 2016.
[33] JVET common test conditions and evaluation procedures for 360° video. Technical report, Joint Video Exploration Team of ITU Telecommunication Standardization Sector, October 2016.
[34] Algorithm descriptions of projection format coversion and video quality metrics in 360Lib. Technical report, Joint Video Exploration Team of ITU Telecommunication Standardization Sector, March 2017.
[35] AHG8: A study on quality impact of line re-sampling rate in EAP. Technical report, Joint Video Exploration Team of ITU Telecommunication Standardization Sector, July 2017.
[36] J. Kim, W. Kim, S. Ahn, J. Kim, and S. Lee. Virtual reality sickness predictor: Anal- ysis of visual-vestibular conflict and VR contents. In Proc. of IEEE International Conference on Quality of Multimedia Experience (QoMEX’18), pages 1–6, Cagliari, Italy, May 2018.
[37] J.LeFeuvreandC.Concolato.Tiled-basedadaptivestreamingusingMPEG-DASH. In Proc. of the 7th ACM on Multimedia Systems Conference (MMSys’16), pages 41:1–41:3, Klagenfurt, Austria, May 2016.
[38] L. Li, Z. Li, X. Ma, H. Yang, and H. Li. Co-projection-plane based 3-D padding for polyhedron projection for 360-degree video. In Proc. of IEEE International Conference on Multimedia and Expo (ICME’17), pages 55–60, Hong Kong, China, July 2017.
[39] R. McGill, J. Tukey, and W. Larsen. Variations of box plots. The American Statisti- cian, 32(1):12–16, February 1978.
[40] MP4Client video player, October 2018. https://gpac.wp.imt.fr/ player/.
[41] Oculus Rift web page, July 2018. https://www.oculus.com/.
[42] Oculus Video web page, July 2018. https://www.oculus.com/
experiences/rift/926562347437041/.
[43] Open-Source Virtual Reality web page, July 2017. https://osvr.github.
io/.
[44] OSVR-Oculus-Rift github, July 2018. https://github.com/OSVR/
OSVR-Oculus-Rift.
[45] C. Ozcinar, A. De Abreu, and A. Smolic. Viewport-aware adaptive 360° video streaming using tiles for virtual reality. In Proc. of IEEE International Conference on Image Processing (ICIP’17), pages 2174–2178, Beijing, China, September 2017.
[46] S. Petrangeli, V. Swaminathan, M. Hosseini, and F. De Turck. An HTTP/2-based adaptive streaming framework for 360° virtual reality videos. In Proc. of ACM on Multimedia Conference (MM’17), pages 306–314, Mountain View, California, USA, October 2017.
[47] S. Petrangeli, V. Swaminathan, M. Hosseini, and F. De Turck. Improving virtual reality streaming using HTTP/2. In Proc. of the 8th ACM on Multimedia Systems Conference (MMSys’17), pages 225–228, Taipei, Taiwan, June 2017.
[48] S. Petrangeli, J. van der Hooft, T. Wauters, R. Huysegems, P. Alface, T. Bostoen, and F. De Turck. Live streaming of 4K ultra-high definition video over the internet. In Proc. of the 7th International Conference on Multimedia Systems (MMSys’16), pages 27:1–27:4, Klagenfurt, Austria, 2016.
[49] M. H. Pinson and S. Wolf. Comparing subjective video quality testing method- ologies. In Visual Communications and Image Processing 2003, pages 573–582. International Society for Optics and Photonics, 2003.
[50] Poke ́mon Go, October 2018. http://www.pokemongo.com/.
[51] Qualinet white paper on definitions of quality of experience, March 2013. https:
//hal.archives-ouvertes.fr/hal-00977812.
[52] U. Reiter, K. Brunnstro ̈m, K. De-Moor, M.-C. Larabi, M. Pereira, A. Pinheiro, J. You, and A. Zgank. Factors influencing quality of experience. In S. Mo ̈ller and A. Raake, editors, Quality of Experience: Advanced Concepts, Applications and Methods, chapter 4, pages 55–72. Springer US, March 2014.
[53] J. Sauer, J. Schneider, and M. Wien. Improved motion compensation for 360° video projected to polytopes. In Proc. of IEEE International Conference on Multimedia and Expo (ICME’17), pages 61–66, Hong Kong, China, July 2017.
[54] R. Schatz, A. Sackl, C. Timmerer, and B. Gardlo. Towards subjective quality of experience assessment for omnidirectional video streaming. In Proc. of IEEE Inter- national Conference on Quality of Multimedia Experience (QoMEX’17), pages 1–6, Erfurt, Germany, May 2017.
[55] R. Silva, B. Feijo ́, P. Gomes, T. Frensh, and D. Monteiro. Real time 360° video stitching and streaming. In Proc of ACM SIGGRAPH 2016 Posters (SIGGRAPH ’16), pages 70:1–70:2, Anaheim, California, July 2016.
[56] A. Singla, S. Fremerey, W. Robitza, P. Lebreton, and A. Raake. Comparison of subjective quality evaluation for HEVC encoded omnidirectional videos at different bit-rates for uhd and fhd resolution. In Proc. of Thematic Workshops of ACM Multi- media (Thematic Workshops’17), pages 511–519, Mountain View, California, USA, October 2017.
[57] A. Singla, S. Fremerey, W. Robitza, and A. Raake. Measuring and comparing QoE and simulator sickness of omnidirectional videos in different head mounted displays. In Proc. of IEEE International Conference on Quality of Multimedia Experience (QoMEX’17), pages 1–6, Erfurt, Germany, May 2017.
[58] K. Skarseth, H. Bjørlo, P. Halvorsen, M. Riegler, and C. Griwodz. OpenVQ: A video quality assessment toolkit. In Proceedings of the 2016 ACM on Multimedia Conference, pages 1197–1200. ACM, 2016.
[59] SKYBOX VR Video Player web page, July 2018. https://www.oculus. com/experiences/gear-vr/1256687334386627/.
[60] W. Song and D. Tjondronegoro. Acceptability-based QoE models for mobile video. IEEE Transactions on Multimedia, 16(3):738–750, April 2014.
[61] W. Song, Y. Xiao, D. Tjondronegoro, and A. Liotta. QoE modelling for VP9 and H.265 videos on mobile devices. In Proc. of ACM international conference on Mul- timedia (MM’15), pages 501–510, Brisbane, Australia, October 2015.
[62] Y. Sun, A. Lu, and L. Yu. Weighted-to-spherically-uniform quality evaluation for omnidirectional video. IEEE Signal Processing Letters, 24(9):1408–1412, Sept 2017.
[63] H. Tran, N. Ngoc, C. Bui, M. Pham, and T. Thang. An evaluation of quality metrics for 360 videos. In Proc. of IEEE 9th International Conference on Ubiquitous and Future Networks (ICUFN’17), pages 7–11, Milan, Italy, July 2017.
[64] H. Tran, N. Ngoc, C. Pham, Y. Jung, and T. Thang. A subjective study on QoE of 360 video for VR communication. In Proc. of IEEE 19th International Workshop on Multimedia Signal Processing (MMSP’17), pages 1–6, Luton, UK, October 2017.
[65] H. Tran, C. Pham, N. Ngoc, A. Pham, and T. Thang. A study on quality metrics for 360 video communications. IEICE TRANSACTIONS on Information and Systems, 101(1):28–36, January 2018.
[66]E.Upenik,M.Rˇeˇra ́bek,andT.Ebrahimi.Testbedforsubjectiveevaluationofomni- directional visual content. In Proc. of Picture Coding Symposium (PCS’16), pages 1–5, Nuremberg, Germany, December 2016.
[67] E. Upenik, M. Rerabek, and T. Ebrahimi. On the performance of objective metrics for omnidirectional visual content. In Proc. of IEEE International Conference on Quality of Multimedia Experience (QoMEX’17), pages 1–6, Erfurt, Germany, May 2017.
[68] Y. Wang, K. Zhu, J. Wu, and Y. Zhu. Content aware video quality prediction model for HEVC encoded bitstream. Multimedia Tools and Applications, 76(18):19191– 19209, March 2017.
[69] Z. Wang, E. P. Simoncelli, and A. C. Bovik. Multiscale structural similarity for image quality assessment. In Signals, Systems and Computers, 2004. Conference Record of the Thirty-Seventh Asilomar Conference on, volume 2, pages 1398–1402. IEEE, 2003.
[70] WebVR: Bringing virtual reality to the web, July 2018. https://webvr. info/.
[71] Whirligig Player web page, July 2018. http://www.whirligig.xyz/.
[72] M. Xiao, S. Wang, C. Zhou, L. Liu, Z. Li, Y. Liu, and S. Chen. Miniview layout for bandwidth-efficient 360-degree video. In Proc. of ACM International Conference on Multimedia (MM’18), pages 914–922, Seoul, Republic of Korea, October 2018.
[73] S. Yao. A 360 degree video player and head movement logger for an HMD us- ing the OSVR API, August 2018. https://github.com/shunhuaiyao/ 360Degree_Head_Movement_Dataset/tree/win32/.
[74] YouTube, August 2018. https://www.youtube.com/.
[75] YouTube Engineering and Developers Blog web page, July 2018. https:// youtube-eng.googleblog.com/2017/03/improving-vr-videos. html.
[76] M. Yu, H. Lakshman, and B. Girod. A framework to evaluate omnidirectional video coding schemes. In Proc. of IEEE International Symposium on Mixed and Aug- mented Reality (ISMAR’15), pages 31–36, Fukuoka, Japan, September 2015.
[77] V. Zakharchenko, K. P. Choi, and J. H. Park. Quality metric for spherical panoramic video. In Proc. of SPIE Optical Engineering+ Applications, pages 99700C–99700C, San Diego, California, United States, September 2016.
[78] A. Zare, A. Aminlou, M. Hannuksela, and M. Gabbouj. HEVC-compliant tile-based streaming of panoramic video for virtual reality applications. In Proc. of ACM In- ternational Conference on Multimedia (MM’16), pages 601–605, Amsterdam, The Netherlands, October 2016.
[79] B. Zhang, J. Zhao, S. Yang, Y. Zhang, J. Wang, and Z. Fei. Subjective and objective quality assessment of panoramic videos in virtual reality environments. In Proc. of IEEE International Conference on Multimedia and Expo Workshops (ICMEW’17), pages 163–168, Hong Kong, China, July 2017.
[80] Y. Zhu, A. Hanjalic, and J. A. Redi. QoE prediction for enriched assessment of individual video viewing experience. In Proc. of ACM international conference on Multimedia (MM’16), pages 801–810, Amsterdam, The Netherlands, October 2016.