ZTE Communications ›› 2025, Vol. 23 ›› Issue (2): 85-95.DOI: 10.12142/ZTECOM.202502009
• Review • Previous Articles Next Articles
LI Zhaoyu1, MENG Xuewei1, ZHANG Jiaqi1, HUANG Cheng2,3(
), JIA Chuanmin1, MA Siwei1, JIANG Yun1
Received:2024-07-09
Online:2025-06-25
Published:2025-06-10
About author:LI Zhaoyu received her BE degree in computer science and technology from Chang’an University， China in 2023. She is currently pursuing her MS degree in computer application technology with Peking University, China. Her research interests include video coding and video coding standard.Supported by:LI Zhaoyu, MENG Xuewei, ZHANG Jiaqi, HUANG Cheng, JIA Chuanmin, MA Siwei, JIANG Yun. Overview of Cross-Component In-Loop Filters in Video Coding Standards[J]. ZTE Communications, 2025, 23(2): 85-95.
Figure 1 Illustration of the ECM video decoder diagram, with golden boxes corresponding to cross-component filters
Figure 2 Illustration of CCALF
Figure 3 Illustration of the relative location of filtered chroma sample of CCALF and its support in the luma channel for 4∶2∶0 chroma format in H.266/VVC
Figure 4 Illustration of the two filter shapes of CCALF in Ref. [45]
Figure 5 Illustration of the filter shape of CCALF with 25 taps
Figure 6 Illustration of CCALF’s shape at ECM-12.0 (the left cross-like filter uses the reconstructed spatial sample of luma sample adaptive offset as input with 23 taps, and the right one uses luma residual samples as input)
Figure 7 Illustration of SAO process when CCSAO is applied
Figure 8 Illustration of the collocated sample used for the CCSAO classification. The left graph shows the 9 locations of the luma component (one of the 9 samples will be chosen based on rate-distortion optimization) and the green and blue samples show the two collocated chroma samples
Figure 9 Illustration of the edge-based classification of CCSAO. Four graphs show four different directions, where the yellow samples are the locations used for calculating the class index at different directional patterns
| Class | AI | RA | ||||
|---|---|---|---|---|---|---|
| Y | Cb | Cr | Y | Cb | Cr | |
| A1 | 0.09% | -1.21% | -3.32% | 0.07% | -1.00% | -3.96% |
| A2 | 0.11% | -2.78% | -3.23% | 0.13% | -2.62% | -4.94% |
| B | 0.12% | -3.35% | -3.22% | 0.15% | -4.31% | -3.41% |
| C | 0.10% | -1.67% | -1.91% | 0.03% | -1.48% | -2.12% |
| E | 0.15% | -3.12% | -2.96% | - | - | - |
| Average | 0.11% | -2.49% | -2.90% | 0.10% | -2.56% | -3.48% |
| D | 0.02% | -0.42% | -0.18% | -0.01% | -0.94% | -0.53% |
| F | 0.10% | -1.77% | -1.07% | 0.15% | -1.08% | -0.32% |
| TGM | 0.12% | -1.19% | -0.72% | 0.16% | -1.26% | -1.03% |
Table 1 Experimental results of ECM-12.0 (anchor: ECM-12.0 without CCALF)
| Class | AI | RA | ||||
|---|---|---|---|---|---|---|
| Y | Cb | Cr | Y | Cb | Cr | |
| A1 | 0.09% | -1.21% | -3.32% | 0.07% | -1.00% | -3.96% |
| A2 | 0.11% | -2.78% | -3.23% | 0.13% | -2.62% | -4.94% |
| B | 0.12% | -3.35% | -3.22% | 0.15% | -4.31% | -3.41% |
| C | 0.10% | -1.67% | -1.91% | 0.03% | -1.48% | -2.12% |
| E | 0.15% | -3.12% | -2.96% | - | - | - |
| Average | 0.11% | -2.49% | -2.90% | 0.10% | -2.56% | -3.48% |
| D | 0.02% | -0.42% | -0.18% | -0.01% | -0.94% | -0.53% |
| F | 0.10% | -1.77% | -1.07% | 0.15% | -1.08% | -0.32% |
| TGM | 0.12% | -1.19% | -0.72% | 0.16% | -1.26% | -1.03% |
| Class | AI | RA | ||||
|---|---|---|---|---|---|---|
| Y | Cb | Cr | Y | Cb | Cr | |
| A1 | -0.28% | -0.83% | -1.36% | -0.42% | -1.89% | -2.42% |
| A2 | 0.01% | -0.99% | -1.15% | -0.06% | -1.88% | -2.01% |
| B | 0.08% | -1.94% | -1.63% | -0.16% | -3.76% | -4.07% |
| C | 0.11% | -0.83% | -0.41% | 0.00% | -2.10% | -1.20% |
| E | 0.02% | -1.55% | -0.68% | - | - | - |
| Average | 0.01% | -1.28% | -1.08% | -0.15% | -2.57% | -2.56% |
| D | 0.03% | -0.02% | -0.31% | 0.10% | -1.56% | -1.05% |
| F | -0.23% | -1.99% | -1.74% | -0.15% | -2.99% | -1.54% |
| TGM | -0.73% | -1.64% | -1.81% | -1.01% | -2.72% | -3.38% |
Table 2 Experimental results of ECM-12.0 (anchor: ECM-12.0 without CCSAO)
| Class | AI | RA | ||||
|---|---|---|---|---|---|---|
| Y | Cb | Cr | Y | Cb | Cr | |
| A1 | -0.28% | -0.83% | -1.36% | -0.42% | -1.89% | -2.42% |
| A2 | 0.01% | -0.99% | -1.15% | -0.06% | -1.88% | -2.01% |
| B | 0.08% | -1.94% | -1.63% | -0.16% | -3.76% | -4.07% |
| C | 0.11% | -0.83% | -0.41% | 0.00% | -2.10% | -1.20% |
| E | 0.02% | -1.55% | -0.68% | - | - | - |
| Average | 0.01% | -1.28% | -1.08% | -0.15% | -2.57% | -2.56% |
| D | 0.03% | -0.02% | -0.31% | 0.10% | -1.56% | -1.05% |
| F | -0.23% | -1.99% | -1.74% | -0.15% | -2.99% | -1.54% |
| TGM | -0.73% | -1.64% | -1.81% | -1.01% | -2.72% | -3.38% |
Figure 10 Illustration of subjective quality comparison. (a)–(c): BasketballDrill, RA configuration and QP22; (d)–(f): BQMall, RA configuration and QP22; (g)–(i): MarketPlace, random access configuration and QP32
| 1 | SULLIVAN G J, OHM J R, HAN W J, et al. Overview of the high efficiency video coding (HEVC) standard [J]. IEEE transactions on circuits and systems for video technology, 2012, 22(12): 1649–1668. DOI: 10.1109/TCSVT.2012.2221191 |
| 2 | WIEGAND T, SULLIVAN G J, BJONTEGAARD G, et al. Overview of the H.264/AVC video coding standard [J]. IEEE transactions on circuits and systems for video technology, 2003, 13(7): 560–576. DOI: 10.1109/TCSVT.2003.815165 |
| 3 | BROSS B, WANG Y K, YE Y, et al. Overview of the versatile video coding (VVC) standard and its applications [J]. IEEE transactions on circuits and systems for video technology, 2021, 31(10): 3736–3764 |
| 4 | COBAN M, LIAO R L, NASER K, et al. Algorithm description of enhanced compression model 12 (ECM 12) [R]. Joint Video Experts Team (JVET), document JVET-AG2025, 2024 |
| 5 | KARCZEWICZ M, HU N, TAQUET J, et al. VVC in-loop filters [J]. IEEE transactions on circuits and systems for video technology, 2021, 31(10): 3907–3925. DOI: 10.1109/tcsvt.2021.3072297 |
| 6 | ANDERSSON K, MISRA K, IKEDA M, et al. Deblocking filtering in VVC [C]//Picture Coding Symposium (PCS). IEEE, 2021: 1–5. DOI: 10.1109/pcs50896.2021.9477477 |
| 7 | FU C M, ALSHINA E, ALSHIN A, et al. Sample adaptive offset in the HEVC standard [J]. IEEE transactions on circuits and systems for video technology, 2012, 22(12): 1755–1764. DOI: 10.1109/TCSVT.2012.2221529 |
| 8 | TSAI C Y, CHEN C Y, YAMAKAGE T, et al. Adaptive loop filtering for video coding [J]. IEEE journal of selected topics in signal processing, 2013, 7(6): 934–945. DOI: 10.1109/JSTSP.2013.2271974 |
| 9 | LU T R, PU F J, YIN P, et al. Luma mapping with chroma scaling in versatile video coding [C]//Data Compression Conference (DCC). IEEE, 2020: 193–202. DOI: 10.1109/DCC47342.2020.00027 |
| 10 | STRÖM J, WENNERSTEN P, ENHORN J, et al. Bilateral loop filter in combination with SAO [C]//Picture Coding Symposium (PCS). IEEE, 2019: 1–5. DOI: 10.1109/PCS48520.2019.8954554 |
| 11 | ZHANG J, JIA C M, ZHANG N, et al. Structure-driven adaptive non-local filter for high efficiency video coding (HEVC) [C]//Data Compression Conference (DCC). IEEE, 2016: 91–100. DOI: 10.1109/DCC.2016.105 |
| 12 | MA S W, ZHANG X F, ZHANG J, et al. Nonlocal in-loop filter: the way toward next-generation video coding? [J]. IEEE multimedia, 2016, 23(2): 16–26. DOI: 10.1109/MMUL.2016.16 |
| 13 | MENG X W, JIA C M, WANG S S, et al. Optimized non-local in-loop filter for video coding [C]//Picture Coding Symposium (PCS). IEEE, 2018: 233–237. DOI: 10.1109/PCS.2018.8456299 |
| 14 | JIA C M, LUO F L, ZHANG X F, et al. Fast non-local adaptive in-loop filter optimization on GPU [J]. IEEE transactions on multimedia, 2020, 23: 39–51. DOI: 10.1109/TMM.2020.2981185 |
| 15 | ZHANG X F, XIONG R Q, LIN W S, et al. Low-rank-based nonlocal adaptive loop filter for high-efficiency video compression [J]. IEEE transactions on circuits and systems for video technology, 2017, 27(10): 2177–2188. DOI: 10.1109/TCSVT.2016.2581618 |
| 16 | MENG X W, JIA C M, ZHANG X F, et al. Parametric non-local in-loop filter for future video coding [C]//Data Compression Conference (DCC). IEEE, 2022: 474. DOI: 10.1109/DCC52660.2022.00085 |
| 17 | MENG X W, JIA C M, ZHANG X F, et al. Deformable Wiener filter for future video coding [J]. IEEE transactions on image processing, 2022, 31: 7222–7236. DOI: 10.1109/TIP.2022.3221278 |
| 18 | WANG Z, MENG X, JIA C, et al. Description of SDR video coding technology proposal by DJI and Peking University [R]. Joint Video Experts Team (JVET), document JVET-J0011, 2018 |
| 19 | MENG X, JIA C, WANG Z, et al. CE2: non-local structure-based filter [R]. Joint Video Experts Team (JVET), document JVET-K0160, 2018 |
| 20 | CHERNYAK R, Victor S, Ikonin S, et al. CE2: noise suppression filter [R]. Joint Video Experts Team (JVET), document JVET-K0053, 2018 |
| 21 | LAI C Y, CHEN C Y, HUANG Y W, et al. CE2.5.2: nonlocal mean in-loop filter [R]. Joint Video Experts Team (JVET), document JVET-K0236, 2018 |
| 22 | ZHANG K, ZUO W M, CHEN Y J, et al. Beyond a Gaussian denoiser: residual learning of deep CNN for image denoising [J]. IEEE transactions on image processing, 2017, 26(7): 3142–3155. DOI: 10.1109/TIP.2017.2662206 |
| 23 | WANG M Z, WAN S, GONG H, et al. Attention-based dual-scale CNN in-loop filter for versatile video coding [J]. IEEE access, 2019, 7: 145214–145226 |
| 24 | CHEN S J, CHEN Z Z, WANG Y B, et al. In-loop filter with dense residual convolutional neural network for VVC [C]//IEEE Conference on Multimedia Information Processing and Retrieval (MIPR). IEEE, 2020: 149-152. DOI: 10.1109/mipr49039.2020.00038 |
| 25 | HUANG Z J, GUO X P, SHANG M Y, et al. An efficient QP variable convolutional neural network based in-loop filter for intra coding [C]//Data Compression Conference (DCC). IEEE, 2021: 33–42. DOI: 10.1109/DCC50243.2021.00011 |
| 26 | LI Y, ZHANG K, LI J, et al. EE1-1.6: deep in-loop filter with fixed point implementation [R]. Joint Video Experts Team (JVET), document JVET-AA0111, 2022 |
| 27 | LI J, LI Y, ZHANG K, et al. EE1-1.6: RDO considering deep in-loop filtering [R]. Joint Video Experts Team (JVET), document JVET-AB0068, 2022 |
| 28 | LIU D, STRÖM J, DAMGHANIAN M, P. et al. EE1-1.5: combined intra and inter models for luma and chroma [R]. Joint Video Experts Team (JVET), document JVET-AC0089, 2023 |
| 29 | LI Y, ZHANG K, ZHANG L. EE1-1.7: deep in-loop filter with additional input information [R]. Joint Video Experts Team (JVET), document JVET-AC0177, 2023 |
| 30 | GOFFMAN-VINOPAL L, PORAT M. Color image compression using inter-color correlation [C]//International Conference on Image Processing. IEEE, 2002: II. DOI: 10.1109/ICIP.2002.1039960 |
| 31 | SONG B C, LEE Y G, KIM N H. Block adaptive inter-color compensation algorithm for RGB 4∶4∶4 video coding [J]. IEEE transactions on circuits and systems for video technology, 2008, 18(10): 1447–1451. DOI: 10.1109/TCSVT.2008.2002827 |
| 32 | LEE S H, CHO N I. Intra prediction method based on the linear relationship between the channels for YUV 4∶2∶0 intra coding [C]//The 16th IEEE International Conference on Image Processing (ICIP). IEEE, 2009: 1037–1040. DOI: 10.1109/icip.2009.5413727 |
| 33 | LI J R, WANG M, ZHANG L, et al. Sub-sampled cross-component prediction for emerging video coding standards [J]. IEEE transactions on image processing, 2021, 30: 7305–7316. DOI: 10.1109/TIP.2021.3104191 |
| 34 | FLYNN D, MARPE D, NACCARI M, et al. Overview of the range extensions for the HEVC standard: tools, profiles, and performance [J]. IEEE transactions on circuits and systems for video technology, 2016, 26(1): 4–19 |
| 35 | PFAFF J, FILIPPOV A, LIU S, et al. Intra prediction and mode coding in VVC [J]. IEEE transactions on circuits and systems for video technology, 2021, 31(10): 3834–3847 |
| 36 | DUAN W H, CHANG Z, JIA C M, et al. Learned image compression using cross-component attention mechanism [J]. IEEE transactions on image processing, 2023, 32: 5478–5493. DOI: 10.1109/TIP.2023.3319275 |
| 37 | CHIEN W J, BOYCE J, CHEN W, et al. JVET AHG report: tool reporting procedure (AHG13) [R]. Joint Video Experts Team (JVET), document JVET-T0013, 2020 |
| 38 | MISRA K, BOSSEN F, SEGALL A. On cross component adaptive loop filter for video compression [C]//Picture Coding Symposium (PCS). IEEE, 2019: 1–5. DOI: 10.1109/PCS48520.2019.8954547 |
| 39 | MISRA K, BOSSEN F, SEGALL A. Cross-component adaptive loop filter for chroma [R]. Joint Video Experts Team (JVET), document JVET-O0636, 2019 |
| 40 | HU N, DONG J, SEREGIN V, et al. CE5-related: reduced filter shape for cross component adaptive loop filter [R]. Joint Video Experts Team (JVET), document JVET-P0558, 2019 |
| 41 | KOTRA A M, ESENLIK S, WANG B, et al. AHG16/CE5-Related: simplifications for cross component adaptive loop filter [R]. Joint Video Experts Team (JVET), document JVET-P0106, 2019 |
| 42 | LI J Y, LIM C S. AHG16/Non-CE5: cross component ALF simplification [R]. Joint Video Experts Team (JVET), document JVET-P0173, 2019 |
| 43 | ZHAO Y, YANG H T. CE5-related: simplified CCALF with 6 filter coefficients [R]. Joint Video Experts Team (JVET), document JVET-P0251, 2019 |
| 44 | KEATING S, BROWNE A, SHARMAN K. AHG12: extensions to CCALF [R]. Joint Video Experts Team(JVET), document JVET-V0080, 2021 |
| 45 | SARWE M G, LIAO R L, CHEN J, et al. AHG12: CTB level filter shape selection of CCALF [R]. Joint Video Experts Team (JVET), document JVET-W0079, 2021 |
| 46 | SARWER M G, LIAO R L, CHEN J, et al. EE2-4.2, EE2-4.3: on CCALF filter [R]. Joint Video Experts Team (JVET), document JVET-X0045, 2021 |
| 47 | HU N, SEREGIN V, KARCZEWICZ M, et al. EE2-4.8: joint tests of chroma BIF, ALF and CCALF [R]. Joint Video Experts Team (JVET), document JVET-X0071, 2021 |
| 48 | YIN W B, ZHANG K, ZHANG L. Non-EE2: extensions of residual-based taps in ALF and CCALF [R]. Joint Video Experts Team (JVET), document JVET-AD0234, 2023 |
| 49 | YIN W B, ZHANG K, DENG Z P, et al. Non-EE2: luma residual taps in chroma-ALF and CCALF [R]. Joint Video Experts Team (JVET), document JVET-AE0121, 2023 |
| 50 | OHM J R. Meeting report of the 32nd Meeting of the Joint Video Experts Team (JVET) [R]. Joint Video Experts Team (JVET), document JVET-AF1000, 2023 |
| 51 | HU N, DONG J, SEREGIN V, et al. CE5-related: multiplication removal for cross component adaptive loop filter [R]. Joint Video Experts Team (JVET), document JVET-P0557, 2019 |
| 52 | STRÖM J, ANDERSSON K. CE5-related: on the CCALF filtering process [R]. Joint Video Experts Team(JVET), document JVET-Q0165, 2020 |
| 53 | MENG X W, ZHENG X Z, WANG S S, et al. CCALF virtual boundary issue for 4:4:4 and 4:2:2 format [R]. Joint Video Experts Team (JVET), document JVET-R0322, Oct. 2019 |
| 54 | MENG X W, ZHENG X Z, WANG S S, et al. AHG 10: One-pass CCALF [R]. Joint Video Experts Team (JVET), document JVET-R0327, Oct. 2019 |
| 55 | YIN W B, ZHANG K, ZHANG L. Non-EE2: coefficient precision adjustment for ALF [R]. Joint Video Experts Team (JVET) document JVET-AF0198, Oct. 2023 |
| 56 | HU N, KARCZEWICZ M, SEREGIN V, et al. EE2-related: adaptive precision for CCALF coefficients [R]. Joint Video Experts Team (JVET), document JVET-AG0233, 2024 |
| 57 | SONG N, YU Y, YU H P, et al. Non-EE2: adaptive coefficient precision for CCALF [R]. Joint Video Experts Team (JVET), document JVET-AG0065, 2024 |
| 58 | KUO C W, XIU X Y, CHEN Y W, et al. Cross-component sample adaptive offset [C]//Data Compression Conference (DCC). IEEE, 2022: 359–368. DOI: 10.1109/DCC52660.2022.00044 |
| 59 | KUO C W, XIU X Y, CHEN W, et al. Cross-component sample adaptive offset [R]. Joint Video Experts Team (JVET), document AVS-M5595, 2020 |
| 60 | KUO C W, XIU X Y, CHEN Y W, et al. EE2-5.1: cross-component sample adaptive offset [R]. Joint Video Experts Team (JVET), document JVET-W0066, 2021 |
| 61 | KOTRA A M, HU N, SEREGIN V, et al. AHG12: edge classifier for cross-component sample adaptive offset (CCSAO) [R]. Joint Video Experts Team (JVET), document JVET-X0105, 2021 |
| 62 | KUO C W, XIU X, CHEN W Y, et al. AHG12: CCSAO classification with edge information [R]. Joint Video Experts Team (JVET), document JVET-X0152, 2021 |
| 63 | JIAN Y R, ZHANG J Q, LI J R, et al. Enhanced cross component sample adaptive offset for AVS3 [C]//International Conference on Visual Communications and Image Processing (VCIP). IEEE, 2021: 1–5. DOI: 10.1109/VCIP53242.2021.9675321 |
| 64 | KUO C W, XIU X Y, CHEN W, et al. AHG12: CCSAO with extended edge classifiers and history offsets [R]. Joint Video Experts Team (JVET), document JVET-AD0218, 2023 |
| 65 | KUO C W, XIU X Y, CHEN W, et al. EE2-5.1: CCSAO with extended edge classifiers and history offsets [R]. Joint Video Experts Team (JVET), document JVET-AE0151, 2023 |
| 66 | KARCZEWICZ M, YE Y. Common test conditions and evaluation procedures for enhanced compression tool testing [R]. Joint Video Experts Team (JVET), document JVET-AF2017, 2023 |
| [1] | YUAN Lang, HUI Chen, WU Yanfeng, LIAO Ronghua, JIANG Feng, GAO Ying. Video Enhancement Network Based on CNN and Transformer [J]. ZTE Communications, 2024, 22(4): 78-88. |
| [2] | ZHANG Qiang, MEI Junjun, GUAN Tao, SUN Zhewen, ZHANG Zixiang, YU Li. Recent Advances in Video Coding for Machines Standard and Technologies [J]. ZTE Communications, 2024, 22(1): 62-76. |
| [3] | ZHONG Min, ZHOU Yimin, LUO Minke, ZUO Wen. Variable Bit Rate Fuzzy Control for Low Delay Video Coding [J]. ZTE Communications, 2017, 15(2): 55-65. |
| [4] | LIN Tao, ZHAO Liping, ZHOU Kailun. Screen Content Coding in HEVC and Beyond [J]. ZTE Communications, 2016, 14(S0): 51-58. |
| [5] | ZHANG Tao, ZHANG Caixia, ZHAO Xin. Review of AVS Audio Coding Standard [J]. ZTE Communications, 2016, 14(2): 56-62. |
| [6] | Shanshe Wang, Falei Luo, Siwei Ma. Overview of the Second Generation AVS Video Coding Standard (AVS2) [J]. ZTE Communications, 2016, 14(1): 3-11. |
| [7] | Bin Li, Jizheng Xu. An Introduction to High Efficiency Video Coding Range Extensions [J]. ZTE Communications, 2016, 14(1): 12-18. |
| [8] | Tao Lin. Screen Content Coding with Primary and Secondary Reference Buffers for String Matching and Copying [J]. ZTE Communications, 2015, 13(4): 53-60. |
| [9] | Siwei Ma, Shiqi Wang, and Wen Gao. AVS 3D Video Coding Technology and System [J]. ZTE Communications, 2012, 10(2): 13-18. |
| Viewed | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
Full text 22
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||
|
Abstract 40
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||
