Lossy Point Cloud Attribute Compression with Subnode-Based Prediction

doi:10.12142/ZTECOM.202304004

ZTE Communications ›› 2023, Vol. 21 ›› Issue (4): 29-37.DOI: 10.12142/ZTECOM.202304004

• Special Topic • Previous Articles Next Articles

Lossy Point Cloud Attribute Compression with Subnode-Based Prediction

YIN Qian¹, ZHANG Xinfeng², HUANG Hongyue¹, WANG Shanshe¹, MA Siwei¹()

^1.School of Computer Science, Peking University, Beijing 100871, China
^2.School of Computer Science and Technology, University of the Chinese Academy of Sciences, Beijing 100049, China

Received:2023-10-07 Online:2023-12-07 Published:2023-12-07
About author:YIN Qian received her MS degree in signal and information processing from University of Electronic Science and Technology of China in 2021. She is currently pursuing a PhD degree in computer science at Peking University, China. She is actively participating in the research work of the Audio Video Coding Standard (AVS) Workgroup of China and Moving Picture Experts Group (MPEG). Her research interests include video and point cloud compression.
ZHANG Xinfeng received his BS degree in computer science from Hebei University of Technology, China in 2007 and PhD degree in computer science from the Institute of Computing Technology, Chinese Academy of Sciences in 2014. From 2014 to 2017, he was a research fellow with the Rapid-Rich Object Search Lab, Nanyang Technological University, Singapore. From October 2017 to October 2018, he was a postdoctoral fellow with the School of Electrical Engineering System, University of Southern California, Los Angeles, USA. From December 2018 to August 2019, he was a research fellow with the Department of Computer Science, City University of Hong Kong, China. He is currently an assistant professor with the School of Computer Science and Technology, University of Chinese Academy of Sciences. He has authored more than 150 refereed journal/conference papers. His research interests include video compression and processing, image/video quality assessment, and 3D point cloud processing.
HUANG Hongyue received his BE degree in communication engineering from Beijing University of Posts and Telecommunications, China in 2015, MS degree in computer science from the Technical University of Berlin (TUB), Germany in 2018, and PhD degree in computer science from the Free University of Brussels (VUB), Belgium in 2021. He is currently a postdoctoral researcher with the National Engineering Research Center of Visual Technology, Peking University, China. His research interests include inventing and optimizing deep-learning-based compression methods for 2D images/videos and 3D visual content such as immersive videos, point clouds, and light field images.
WANG Shanshe received his BS degree from the Department of Mathematics, Heilongjiang University, China in 2004, MS degree in computer software and theory from Northeast Petroleum University, China in 2010, and PhD degree in computer science from the Harbin Institute of Technology, China. He held a postdoctoral position with Peking University, China from 2016 to 2018. He is currently an associate researcher with the School of Electronics Engineering and Computer Science, Institute of Digital Media, Peking University. His current research interests include video compression and image and video quality assessment.
MA Siwei (swma@stu.pku.edu.cn) received his BS degree from Shandong Normal University, China in 1999, and PhD degree in computer science from the Institute of Computing Technology, Chinese Academy of Sciences in 2005. He held a postdoctoral position with the University of Southern California, Los Angeles, USA from 2005 to 2007. He is currently a professor with the School of Electronics Engineering and Computer Science, Institute of Digital Media, Peking University, China. He has authored over 300 technical articles in refereed journals and proceedings in image and video coding, video processing, video streaming, and transmission. He served/serves as an associate editor for the IEEE Transactions on Circuits and Systems for Video Technology and Journal of Visual Communication and Image Representation.
Supported by:
China Postdoctoral Science Foundation(2022M720234);the National Natural Science Foundation(62071449)

Abstract

Abstract:

Recent years have witnessed that 3D point cloud compression (PCC) has become a research hotspot both in academia and industry. Especially in industry, the Moving Picture Expert Group (MPEG) has actively initiated the development of PCC standards. One of the adopted frameworks called geometry-based PCC (G-PCC) follows the architecture of coding geometry first and then coding attributes, where the region adaptive hierarchical transform (RAHT) method is introduced for the lossy attribute compression. The upsampled transform domain prediction in RAHT does not sufficiently explore the attribute correlations between neighbor nodes and thus fails to further reduce the attribute redundancy between neighbor nodes. In this paper, we propose a subnode-based prediction method, where the spatial position relationship between neighbor nodes is fully considered and prediction precision is further promoted. We utilize some already-encoded neighbor nodes to facilitate the upsampled transform domain prediction in RAHT by means of a weighted average strategy. Experimental results have illustrated that our proposed attribute compression method shows better rate-distortion (R-D) performance than the latest MPEG G-PCC (both on reference software TMC13-v22.0 and GeS-TM-v2.0).

Key words: point cloud compression, MPEG G-PCC, RAHT, subnode-based prediction

YIN Qian, ZHANG Xinfeng, HUANG Hongyue, WANG Shanshe, MA Siwei. Lossy Point Cloud Attribute Compression with Subnode-Based Prediction[J]. ZTE Communications, 2023, 21(4): 29-37.

Figures/Tables 9

References 31

1	TULVAN C, MEKURIA R, LI Z. Use cases for point cloud compression (PCC), output document N16331 [R]. Geneva, Switzerland: ISO/IEC JTC 1/SC29/WG 11 MPEG, 2016
2	MEKURIA R, BLOM K, Design CESAR P., implementation, and evaluation of a point cloud codec for tele-immersive video [J]. IEEE transactions on circuits and systems for video technology, 2017, 27(4): 828–842. DOI: 10.1109/TCSVT.2016.2543039
3	MPEG 3D Graphics Coding Group. Call for proposals for point cloud coding v2, output document N16763 [R]. 2017
4	MPEG 3D Graphics and Haptics Coding Group. V-PCC test model v22, output document N00572 [R]. Antalya, Turkish: ISO/IEC JTC 1/SC 29/WG 11 MPEG, 2023
5	MPEG 3D Graphics and Haptics Coding Group. G-PCC test model v22, output document N00571 [R]. Antalya, Turkish: ISO/IEC JTC 1/SC 29/WG 11 MPEG, 2023
6	SCHNABEL R, KLEIN R. Octree-based point-cloud compression [C]//Symposium on Point-Based Graphics. Eurographics Association, 2006: 111–120. DOI: 10.2312/SPBG/SPBG06/111-120
7	PENG J L, KUO C C J. Progressive geometry encoder using octree-based space partitioning [C]//2004 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 2004: 1–4. DOI: 10.1109/ICME.2004.1394110
8	NAKAGAMI O. Report on triangle soup decoding, input document m52279 [R]. Brussels, Belgium: ISO/IEC JTC 1/SC 29/WG 11 MPEG, 2020
9	FLYNN D, TOURAPIS A, MAMMOU K. Predictive geometry coding, input document m51012 [R]. Geneva, Switzerland: ISO/IEC JTC 1/SC 29/WG11 MPEG, 2019
10	MAMMOU K. PCC test model category 3 v0, output document N17249 [R]. Macau, China: ISO/IEC JTC 1/SC 29/WG 11 MPEG, 2017
11	CHOU P A, DE QUEIROZ R. L. Transform coder for point cloud attributes, input document m38674 [R]. Geneva, Switzerland: ISO/IEC JTC 1/SC29/WG 11 MPEG, 2016
12	DE QUEIROZ R L, CHOU P A. Compression of 3D point clouds using a region-adaptive hierarchical transform [J]. IEEE transactions on image processing, 2016, 25(8): 3947–3956. DOI: 10.1109/TIP.2016.2575005
13	FLYNN D, LASSERRE S. G-PCC CE13.18 report on upsampled transform domain prediction in RAHT, input document m49380 [R]. Gothenburg, Sweden: ISO/IEC JTC 1/SC 29/WG 11 MPEG, 2019
14	WANG W, XU Y, ZHANG Ket al. Sub-node-based prediction in transform domain for RAHT, input document m60203 [R]. Online: ISO/IEC JTC 1/SC 29/WG 11 MPEG, 2022
15	MAMMOU K. Point cloud compression core experiment 13.6 on attributes prediction strategies, output document N18007 [R]. Macau, China: ISO/IEC JTC 1/SC 29/WG 11 MPEG, 2018
16	MAMMOU K, TOURAPIS A, KIM J, et al. Proposal for improved lossy compression in TMC1, input document m42640 [R]. San Diego, United states: ISO/IEC JTC 1/SC 29/WG 11 MPEG, 2018
17	WEI H L, SHAO Y T, WANG J, et al. Enhanced intra prediction scheme in point cloud attribute compression [C]//2019 IEEE Visual Communications and Image Processing (VCIP). IEEE, 2019: 1–4. DOI: 10.1109/VCIP47243.2019.8966001
18	YEA S. VOSOUGHI A, LIU S. Bilateral filtering for predictive transform in G-PCC, input document m46365 [R]. Marrakech, Morocco: ISO/IEC JTC1/SC 29/WG 11 MPEG, 2019
19	YIN Q, REN Q S, ZHAO L L, et al. Lossless point cloud attribute compression with normal-based intra prediction [C]//2021 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB). IEEE, 2021: 1–5. DOI: 10.1109/BMSB53066.2021.9547021
20	ZHANG X M, WAN W G, AN X D. Clustering and DCT based color point cloud compression [J]. Journal of signal processing systems, 2017, 86(1): 41–49. DOI: 10.1007/s11265-015-1095-0
21	COHEN R A, TIAN D, VETRO A. Point cloud attribute compression using 3-D intra prediction and shape-adaptive transforms [C]//2016 Data Compression Conference (DCC). IEEE, 2016: 141–150. DOI: 10.1109/DCC.2016.67
22	WANG L J, WANG L Y, LUO Y T, et al. Point-Cloud compression using data independent method—a 3D discrete cosine transform approach [C]//2017 IEEE International Conference on Information and Automation (ICIA). IEEE, 2017: 1–6. DOI: 10.1109/icinfa.2017.8078873
23	ZHANG C, FLORÊNCIO D, LOOP C. Point cloud attribute compression with graph transform [C]//2014 IEEE International Conference on Image Processing (ICIP). IEEE, 2014: 2066–2070. DOI: 10.1109/ICIP.2014.7025414
24	SHAO Y T, ZHANG Z B, LI Z, et al. Attribute compression of 3D point clouds using Laplacian sparsity optimized graph transform [C]//2017 IEEE Visual Communications and Image Processing (VCIP). IEEE, 2017: 1–4. DOI: 10.1109/VCIP.2017.8305131
25	XU Y Q, HU W, WANG S S, et al. Cluster-based point cloud coding with normal weighted graph Fourier transform [C]//2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2018: 1753–1757. DOI: 10.1109/ICASSP.2018.8462684
26	XU Y Q, HU W, WANG S S, et al. Predictive generalized graph Fourier transform for attribute compression of dynamic point clouds [J]. IEEE transactions on circuits and systems for video technology, 2021, 31(5): 1968–1982. DOI: 10.1109/TCSVT.2020.3015901
27	HOODA R, PAN W D. Early termination of dyadic region-adaptive hierarchical transform for efficient attribute compression of 3D point clouds [J]. IEEE signal processing letters, 2022, 29: 214–218. DOI: 10.1109/LSP.2021.3133204
28	MPEG 3D Graphics and Haptics Coding Group. TMC13 software repository [EB/OL]. (2023-06-02)[2023-11-20].
29	MPEG 3D Graphics and Haptics Coding Group. GeS-TM software repository [EB/OL]. (2023-06-05)[2023-11-20].
30	MPEG 3D Graphics and Haptics Coding Group. Common test conditions for G-PCC, output document N00578 [R]. Antalya, Turkish: ISO/IEC JTC 1/SC29/WG 11 MPEG, 2023
31	MPEG 3D Graphics and Haptics Coding Group. MPEG content repository [EB/OL]. (2023-03-17)[2023-11-20].

G-PCC Platform	Conditions		Datasets
G-PCC Platform	C1	C2	Cat1	Cat2	Cat3
TMC13	√	√	√		√
GeS-TM	√	√		√

G-PCC Platform	Conditions		Datasets
G-PCC Platform	C1	C2	Cat1	Cat2	Cat3
TMC13	√	√	√		√
GeS-TM	√	√		√

Dataset Category	C1 End-to-End BD-Attribute Rate/%				C2 End-to-End BD-Attribute Rate/%
Dataset Category	Luma	Chroma Cb	Chroma Cr	Reflectance	Luma	Chroma Cb	Chroma Cr	Reflectance
Solid average	-0.4	-0.3	-0.4	/	-0.2	-0.3	-0.2	/
Dense average	-0.2	-0.2	-0.2	/	-0.2	-0.5	-0.1	/
Sparse average	-0.2	-0.2	-0.1	/	-0.2	-0.1	-0.3	/
Scant average	-0.2	-0.2	-0.3	/	-0.2	-0.3	-0.2	/
Am-fused average	-0.3	-1.2	-1.1	-1.1	-0.1	-0.6	-0.7	-0.2
Am-frame spinning average	/	/	/	-0.3	/	/	/	-0.2
Am-frame non-spinning average	/	/	/	-0.6	/	/	/	-0.2
Overall average	-0.2	-0.3	-0.3	-0.5	-0.2	-0.3	-0.2	-0.2
Average encoding/decoding time/%	102/103				100/107

Dataset Category	C1 End-to-End BD-Attribute Rate/%				C2 End-to-End BD-Attribute Rate/%
Dataset Category	Luma	Chroma Cb	Chroma Cr	Reflectance	Luma	Chroma Cb	Chroma Cr	Reflectance
Solid average	-0.4	-0.3	-0.4	/	-0.2	-0.3	-0.2	/
Dense average	-0.2	-0.2	-0.2	/	-0.2	-0.5	-0.1	/
Sparse average	-0.2	-0.2	-0.1	/	-0.2	-0.1	-0.3	/
Scant average	-0.2	-0.2	-0.3	/	-0.2	-0.3	-0.2	/
Am-fused average	-0.3	-1.2	-1.1	-1.1	-0.1	-0.6	-0.7	-0.2
Am-frame spinning average	/	/	/	-0.3	/	/	/	-0.2
Am-frame non-spinning average	/	/	/	-0.6	/	/	/	-0.2
Overall average	-0.2	-0.3	-0.3	-0.5	-0.2	-0.3	-0.2	-0.2
Average encoding/decoding time/%	102/103				100/107

Dataset Category	C1 BD-Rate/%			C2 BD-Rate/%
Dataset Category	L	Cb	Cr	L	Cb	Cr
Cat2-A average	-0.4	-0.5	-0.5	-0.3	-0.3	-0.4
Cat2-B average	-0.3	-0.3	-0.3	-0.2	-0.2	-0.2
Cat2-C average	-0.5	-0.4	-0.5	-0.4	-0.4	-0.3
Overall average	-0.4	-0.4	-0.5	-0.3	-0.3	-0.4
Avgerage encoding/decoding time (%)	106/109			101/110

Lossy Point Cloud Attribute Compression with Subnode-Based Prediction

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 9

References 31

Related Articles 1

Recommended Articles

Metrics