Beyond Video Quality: Evaluation of Spatial Presence in 360-Degree Videos

doi:10.12142/ZTECOM.202304012

ZTE Communications ›› 2023, Vol. 21 ›› Issue (4): 91-103.DOI: 10.12142/ZTECOM.202304012

收稿日期:2023-05-08 出版日期:2023-12-25 发布日期:2023-12-07

Beyond Video Quality: Evaluation of Spatial Presence in 360-Degree Videos

ZOU Wenjie¹, GU Chengming¹, FAN Jiawei¹, HUANG Cheng²^,³(), BAI Yaxian²^,³

^1.School of Telecommunications Engineering, Xidian University, Xi’an 710071, China
^2.ZTE Corporation, Shenzhen 518057, China
^3.State Key Laboratory of Mobile Network and Mobile Multimedia Technology, Shenzhen 518055, China

Received:2023-05-08 Online:2023-12-25 Published:2023-12-07
About author:ZOU Wenjie received his BS and PhD degrees from Xidian University, China in 2009 and 2017, respectively. He is currently a lecturer with the Multimedia Communication Laboratory, Xidian University. His research interests include QoE, video quality assessment, and multimedia compression.
GU Chengming received his BE degree in communication engineering from Xidian University, China in 2021. He is currently working toward an ME degree in information and communication engineering with Xidian University. His research interests include video coding and processing.
FAN Jiawei received his BE degree in telecommunications engineering from Xidian University, China in 2022. He is currently working toward his ME degree in electronic information with Xidian University. His research interests include video coding and processing.
HUANG Cheng (huang.cheng5@zte.com.cn) received his MS degree from the School of Computer Science and Engineering, Southeast University, China. He is currently a senior system architect and project manager of video technology research at ZTE Corporation. His research interests include visual coding, storage, transport, and multimedia systems.
BAI Yaxian received his MS degree in communication engineering from Wuhan University of Technology, China. She is currently a senior engineer at ZTE Corporation. Her research interests include video coding and processing and point cloud compression and transmission.
Supported by:
ZTE Industry?University?Institute Cooperation Funds

摘要/Abstract

Abstract:

With the rapid development of immersive multimedia technologies, 360-degree video services have quickly gained popularity and how to ensure sufficient spatial presence of end users when viewing 360-degree videos becomes a new challenge. In this regard, accurately acquiring users’ sense of spatial presence is of fundamental importance for video service providers to improve their service quality. Unfortunately, there is no efficient evaluation model so far for measuring the sense of spatial presence for 360-degree videos. In this paper, we first design an assessment framework to clarify the influencing factors of spatial presence. Related parameters of 360-degree videos and head-mounted display devices are both considered in this framework. Well-designed subjective experiments are then conducted to investigate the impact of various influencing factors on the sense of presence. Based on the subjective ratings, we propose a spatial presence assessment model that can be easily deployed in 360-degree video applications. To the best of our knowledge, this is the first attempt in literature to establish a quantitative spatial presence assessment model by using technical parameters that are easily extracted. Experimental results demonstrate that the proposed model can reliably predict the sense of spatial presence.

Key words: virtual reality, quality assessment, omnidirectional video, spatial presence

. [J]. ZTE Communications, 2023, 21(4): 91-103.

ZOU Wenjie, GU Chengming, FAN Jiawei, HUANG Cheng, BAI Yaxian. Beyond Video Quality: Evaluation of Spatial Presence in 360-Degree Videos[J]. ZTE Communications, 2023, 21(4): 91-103.

图/表 24

参考文献 46

1	DELOITTE. Digital democracy survey: a multi-generational view of consumer technology, media and telecom trends, digital democracy survey 9th edition [R]. 2022
2	ZHU W H, ZHAI G T, TAO M X, et al. Quality of experience estimation of ultra‑high definition content [J]. ZTE technology journal, 2021, 27(1): 37–43. DOI: 10.12142/ZTETJ.202101009
3	LI J L, ZHAO X, YANG Y. A review of interactive video quality assessment methods [J]. ZTE technology journal, 2021,27(1):44–47.DOI:10.12142/ZTETJ.202101010
4	LOMBARD M, JONES M T. Defining presence [M]//Immersed in media. Cham: Springer International Publishing, 2015: 13–34. DOI: 10.1007/978-3-319-10190-3_2
5	SCHUEMIE M J, VAN DER STRAATEN P, KRIJN M, et al. Research on presence in virtual reality: a survey [J]. CyberPsychology & behavior, 2001, 4(2): 183–201. DOI: 10.1089/109493101300117884
6	SEO Y, KIM M, JUNG Y, et al. Avatar face recognition and self-presence [J]. Computers in human behavior, 2017, 69: 120–127. DOI: 10.1016/j.chb.2016.12.020
7	FELTON W M, JACKSON R E. Presence: a review [J]. International journal of human-computer interaction, 2022, 38(1): 1–18. DOI: 10.1080/10447318.2021.1921368
8	LOMBARD M, DITTON T. At the heart of it all: the concept of presence [J]. Journal of computer-mediated communication, 2006, 3(2). DOI: 10.1111/j.1083-6101.1997.tb00072.x
9	NORTH M M, NORTH S M. A comparative study of sense of presence of virtual reality and immersive environments [J]. Australasian journal of information systems, 2016, 20. DOI: 10.3127/ajis.v20i0.1168
10	GONÇALVES G, MELO M, BARBOSA L, et al. Evaluation of the impact of different levels of self-representation and body tracking on the sense of presence and embodiment in immersive VR [J]. Virtual reality, 2022, 26(1): 1–14. DOI: 10.1007/s10055-021-00530-5
11	SKARBEZ R, BROOKS Jr F P, WHITTON M C. A survey of presence and related concepts [J]. ACM computing surveys (CSUR), 2017, 50(6): 1–39. DOI: 10.1145/3134301
12	LAARNI J, RAVAJA N, SAARI T, et al. Ways to measure spatial presence: review and future directions [M]//Immersed in media. Cham: Springer International Publishing, 2015: 139–185. DOI: 10.1007/978-3-319-10190-3_8
13	AL-JUNDI H A, TANBOUR E Y. A framework for fidelity evaluation of immersive virtual reality systems [J].Virtual reality, 2022, 26(3): 1103–1122. DOI: 10.1007/s10055-021-00618-y
14	EGAN D, BRENNAN S, BARRETT J, et al. An evaluation of heart rate and electrodermal activity as an objective QoE evaluation method for immersive virtual reality environments [C]//Proc. 2016 Eighth International Conference on Quality of Multimedia Experience (QoMEX). IEEE, 2016: 1–6. DOI: 10.1109/QoMEX.2016.7498964
15	CHESSA M, MAIELLO G, BORSARI A, et al. The perceptual quality of the oculus rift for immersive virtual reality [J]. Human-computer interaction, 2019, 34(1): 51–82. DOI: 10.1080/07370024.2016.1243478
16	TERKILDSEN T, MAKRANSKY G. Measuring presence in video games: an investigation of the potential use of physiological measures as indicators of presence [J]. International journal of human-computer studies, 2019, 126: 64–80. DOI: 10.1016/j.ijhcs.2019.02.006
17	GRASSINI S, LAUMANN K. Questionnaire measures and physiological correlates of presence: a systematic review [J]. Frontiers in psychology, 2020, 11: 349. DOI: 10.3389/fpsyg.2020.00349
18	ZOU W J, YANG F Z, ZHANG W, et al. A framework for assessing spatial presence of omnidirectional video on virtual reality device [J]. IEEE access, 2018, 6: 44676–44684. DOI: 10.1109/ACCESS.2018.2864872
19	WITMER B G, SINGER M J. Measuring presence in virtual environments: a presence questionnaire [J]. Presence: teleoperators and virtual environments, 1998, 7(3): 225–240. DOI: 10.1162/105474698565686
20	LESSITER J, FREEMAN J, KEOGH E, et al. A cross-media presence questionnaire: the ITC-sense of presence inventory [J]. Presence: teleoperators and virtual environments, 2001, 10(3): 282–297. DOI: 10.1162/105474601300343612
21	JENNETT C, COX A L, CAIRNS P, et al. Measuring and defining the experience of immersion in games [J]. International journal of human-computer studies, 2008, 66(9): 641–661. DOI: 10.1016/j.ijhcs.2008.04.004
22	VORDERER P, WIRTH W, GOUVEIA F R, et al. MEC spatial presence questionnaire (MEC-SPQ): short documentation and instructions for application [R]. 2004
23	CUMMINGS J J, WERTZ E E. Capturing social presence: Concept explication through an empirical analysis of social presence measures [J]. Journal of computer-mediated communication, 2022, 28(1): zmac027. DOI: 10.1093/jcmc/zmac027
24	LIN J J W, DUH H B L, PARKER D E, et al. Effects of field of view on presence, enjoyment, memory, and simulator sickness in a virtual environment [C]//Proc. IEEE Virtual Reality. IEEE, 2002: 164–171. DOI: 10.1109/VR.2002.996519
25	YU M, LAKSHMAN H, GIROD B. A framework to evaluate omnidirectional video coding schemes [C]//Proc. 2015 IEEE International Symposium on Mixed and Augmented Reality. IEEE, 2015: 31–36. DOI: 10.1109/ISMAR.2015.12
26	ZAKHARCHENKO V, CHOI K P, PARK J H. Quality metric for spherical panoramic video [C]//Proc. SPIE Optics and Photonics for Information Processing X. SPIE, 2016. DOI: 10.1117/12.2235885
27	XU M, LI C, LIU Y F, et al. A subjective visual quality assessment method of panoramic videos [C]//Proc. 2017 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 2017: 517–522. DOI: 10.1109/ICME.2017.8019351
28	UPENIK E, ŘEŘÁBEK M, EBRAHIMI T. Testbed for subjective evaluation of omnidirectional visual content [C]//Proc. 2016 Picture Coding Symposium (PCS). IEEE, 2017: 1–5. DOI: 10.1109/PCS.2016.7906378
29	SCHATZ R, SACKL A, TIMMERER C, et al. Towards subjective quality of experience assessment for omnidirectional video streaming [C]//Proc. Ninth International Conference on Quality of Multimedia Experience (QoMEX). IEEE, 2017: 1–6. DOI: 10.1109/QoMEX.2017.7965657
30	OZCINAR C, CABRERA J, SMOLIC A. Visual attention-aware omnidirectional video streaming using optimal tiles for virtual reality [J]. IEEE journal on emerging and selected topics in circuits and systems, 2019, 9(1): 217–230. DOI: 10.1109/JETCAS.2019.2895096
31	GHAZNAVI-YOUVALARI R, ZARE A, AMINLOU A, et al. Shared coded picture technique for tile-based viewport-adaptive streaming of omnidirectional video [J]. IEEE transactions on circuits and systems for video technology, 2019, 29(10): 3106–3120. DOI: 10.1109/TCSVT.2018.2874179
32	DUAN H Y, ZHAI G T, MIN X K, et al. Perceptual quality assessment of omnidirectional images [C]//IEEE International Symposium on Circuits and Systems (ISCAS). 2018: 1–5. DOI: 10.1109/ISCAS.2018.8351786
33	XU M, LI C, CHEN Z Z, et al. Assessing visual quality of omnidirectional videos [J]. IEEE transactions on circuits and systems for video technology, 2019, 29(12): 3516–3530. DOI: 10.1109/TCSVT.2018.2886277
34	ERMI L, MÄYRÄ F. Fundamental components of the gameplay experience: analysing immersion [C]//Digital Games Research Conference 2005, Changing Views: Worlds in Play. DBLP, 2005: 37–53
35	SLATER M, WILBUR S. A framework for immersive virtual environments (FIVE): speculations on the role of presence in virtual environments [J]. Presence: teleoperators and virtual environments, 1997, 6(6): 603–616. DOI: 10.1162/pres.1997.6.6.603
36	CUMMINGS J J, BAILENSON J N. How immersive is enough? A meta-analysis of the effect of immersive technology on user presence [J]. Media psychology, 2016, 19(2): 272–309. DOI: 10.1080/15213269.2015.1015740
37	ZHAO J B, ALLISON R S, VINNIKOV M, et al. Estimating the motion-to-photon latency in head mounted displays [C]//Proc. 2017 IEEE Virtual Reality (VR). IEEE, 2017: 313–314. DOI: 10.1109/VR.2017.7892302
38	ITU. Methods for the subjective assessment of video quality, audio quality and audiovisual quality of internet video and distribution quality television in any environment: recommendation P.913 [S]. 2021
39	ASBUN E, HE Y, HE Y, et al. AHG8: interdigital test sequences for virtual reality video coding [R]. 2016
40	SUN W, GUO R. Test sequences for virtual reality video coding from letinVR [R]. 2016
41	MERCAT A, VIITANEN M, VANNE J. UVG dataset: 50/120 fps 4K sequences for video codec analysis and development [C]//Proc. ACM Multimedia Systems Conference. ACM, 2020: 297–302. DOI: 10.1145/3339825.3394937
42	ITU. Methods for objective measurements of perceived audio quality: recommendation BS 13871 [S]. 2001
43	BAILENSON J N, SWINTH K, HOYT C, et al. The independent and interactive effects of embodied-agent appearance and behavior on self-report, cognitive, and behavioral markers of copresence in immersive virtual environments [J]. Presence: teleoperators and virtual environments, 2005, 14(4): 379–393. DOI: 10.1162/105474605774785235
44	OU Y F, MA Z, LIU T, et al. Perceptual quality assessment of video considering both frame rate and quantization artifacts [J]. IEEE transactions on circuits and systems for video technology, 2011, 21(3): 286–298. DOI: 10.1109/TCSVT.2010.2087833
45	OU Y F, LIU T, ZHAO Z, et al. Modeling the impact of frame rate on perceptual quality of video [C]//Proc. 15th IEEE International Conference on Image Processing. IEEE, 2008: 689–692. DOI: 10.1109/ICIP.2008.4711848
46	ZOU W J, YANG F Z, WAN S. Perceptual video quality metric for compression artefacts: from two-dimensional to omnidirectional [J]. IET image processing, 2018, 12(3): 374–381. DOI: 10.1049/iet-ipr.2017.0826

BPP	Bitrate/(Mbit/s)			BPP	Bitrate/(Mbit/s)
BPP	720P (1 280×640)	1080P (1 920×960)	2K (2 160×1 080)	BPP	4K (3 840×1 920)
0.016	0.39	0.89	1.12	0.011	2.50
0.032	0.79	1.77	2.24	0.024	5.20
0.056	1.38	3.10	3.92	0.056	12.39
0.08	1.97	4.42	5.60	0.08	17.70
0.16	3.93	8.85	11.20	0.16	35.39
0.20	4.92	11.06	14.00	0.20	44.24

BPP	Bitrate/(Mbit/s)			BPP	Bitrate/(Mbit/s)
BPP	720P (1 280×640)	1080P (1 920×960)	2K (2 160×1 080)	BPP	4K (3 840×1 920)
0.016	0.39	0.89	1.12	0.011	2.50
0.032	0.79	1.77	2.24	0.024	5.20
0.056	1.38	3.10	3.92	0.056	12.39
0.08	1.97	4.42	5.60	0.08	17.70
0.16	3.93	8.85	11.20	0.16	35.39
0.20	4.92	11.06	14.00	0.20	44.24

File Name	Signal Type	File Name	Signal Type
FCODSB1.WAV	music	LCODPIP.WAV	music
GCODCLA.WAV	music	NCODSFE.WAV	speech
LCODHRP.WAV	music	KREFSME.WAV	speech

File Name	Signal Type	File Name	Signal Type
FCODSB1.WAV	music	LCODPIP.WAV	music
GCODCLA.WAV	music	NCODSFE.WAV	speech
LCODHRP.WAV	music	KREFSME.WAV	speech

No.	Video /(Mbit/s)	Audio / (kbit/s)	No.	Video /(Mbit/s)	Audio / (kbit/s)
S1	2	16	S2	8	16
S1	8	32	S2	18	32
S1	18	64	S2	2	64
S1	18	16	S3	2	32
S1	2	64	S3	8	64
S1	4	128	S3	18	128
S2	2	16	S3	8	16
S2	8	64	S3	18	32
S2	18	128	S3	2	128

Beyond Video Quality: Evaluation of Spatial Presence in 360-Degree Videos

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 24

参考文献 46

相关文章 0

编辑推荐

Metrics

No.	Video /(Mbit/s)	Audio / (kbit/s)	No.	Video /(Mbit/s)	Audio / (kbit/s)
S1	2	16	S2	8	16
S1	8	32	S2	18	32
S1	18	64	S2	2	64
S1	18	16	S3	2	32
S1	2	64	S3	8	64
S1	4	128	S3	18	128
S2	2	16	S3	8	16
S2	8	64	S3	18	32
S2	18	128	S3	2	128

No.	Video /(Mbit/s)	Audio / (kbit/s)	No.	Video /(Mbit/s)	Audio / (kbit/s)
S1	2	16	S2	8	16
S1	8	32	S2	18	32
S1	18	64	S2	2	64
S1	18	16	S3	2	32
S1	2	64	S3	8	64
S1	4	128	S3	18	128
S2	2	16	S3	8	16
S2	8	64	S3	18	32
S2	18	128	S3	2	128

Video Resolution	v₄	v₅
720P	0.497 4	0.525 7
1080P	0.529 2	1.369 0
2K	0.537 1	4.584 0
4K	0.497 4	16.580 0

Video (BPP)	Audio/(kbit/s)
0.02	16, 64, 128, 256
0.06	16, 64, 128, 256
0.14	16, 64, 128, 256
0.19	16, 64, 128, 256

Experiment	Subject No. 1			Subject No. 2			Subject No. 3
Experiment	PCC	SROCC	RMSE	PCC	SROCC	RMSE	PCC	SROCC	RMSE
1	0.882	0.878	0.443	0.926	0.922	0.227	0.908	0.902	0.282
2	0.886	0.881	0.462	0.924	0.918	0.214	0.904	0.898	0.343

Experiment	PCC	SROCC	RMSE
1	0.910	0.894	0.277
2	0.908	0.900	0.335

Experiment	PCC	SROCC	RMSE
1	0.910	0.894	0.277
2	0.908	0.900	0.335

No.	Video /(Mbit/s)	Audio / (kbit/s)	No.	Video /(Mbit/s)	Audio / (kbit/s)
S1	2	16	S2	8	16
S1	8	32	S2	18	32
S1	18	64	S2	2	64
S1	18	16	S3	2	32
S1	2	64	S3	8	64
S1	4	128	S3	18	128
S2	2	16	S3	8	16
S2	8	64	S3	18	32
S2	18	128	S3	2	128

No.	Video /(Mbit/s)	Audio / (kbit/s)	No.	Video /(Mbit/s)	Audio / (kbit/s)
S1	2	16	S2	8	16
S1	8	32	S2	18	32
S1	18	64	S2	2	64
S1	18	16	S3	2	32
S1	2	64	S3	8	64
S1	4	128	S3	18	128
S2	2	16	S3	8	16
S2	8	64	S3	18	32
S2	18	128	S3	2	128

No.	Video /(Mbit/s)	Audio / (kbit/s)	No.	Video /(Mbit/s)	Audio / (kbit/s)
S1	2	16	S2	8	16
S1	8	32	S2	18	32
S1	18	64	S2	2	64
S1	18	16	S3	2	32
S1	2	64	S3	8	64
S1	4	128	S3	18	128
S2	2	16	S3	8	16
S2	8	64	S3	18	32
S2	18	128	S3	2	128

No.	Video /(Mbit/s)	Audio / (kbit/s)	No.	Video /(Mbit/s)	Audio / (kbit/s)
S1	2	16	S2	8	16
S1	8	32	S2	18	32
S1	18	64	S2	2	64
S1	18	16	S3	2	32
S1	2	64	S3	8	64
S1	4	128	S3	18	128
S2	2	16	S3	8	16
S2	8	64	S3	18	32
S2	18	128	S3	2	128