Steel Surface Anomaly Detection Using 3D Depth and 2D RGB Features

doi:10.12142/ZTECOM.202601011

ZTE Communications ›› 2026, Vol. 24 ›› Issue (1): 81-87.DOI: 10.12142/ZTECOM.202601011

• Special Topic • Previous Articles Next Articles

Steel Surface Anomaly Detection Using 3D Depth and 2D RGB Features

Zheng Wangguandong¹, Lu Ping², Deng Fangwei², Huang Shijun², Xia Siyu¹()

^1.Southeast University, Nanjing 210096, China
^2.ZTE Corporation, Shenzhen 518057, China

Received:2024-09-27 Online:2026-03-25 Published:2026-03-17
About author:Zheng Wangguandong is pursuing his master’s degree at the School of Automation, Southeast University, China. His research interests focus on artificial intelligence and computer vision, with a specific specialization in image and video generation. He has published four CCF-A conference papers and possesses extensive research experience in image segmentation and object detection.
Lu Ping is the executive deputy director of the National Key Laboratory of Mobile Networks and Mobile Multimedia Technology, China. His research areas encompass cloud computing, big data, augmented reality, and multimedia serviceization. He leads and participates in major national science and technology projects and national science and technology support programs. He has published numerous academic papers and is the author of the books Internet of Things Capability Development and Application and Big Data Technology and Application in Cloud Computing.
Deng Fangwei is a senior strategic planner at ZTE Corporation, specializing in industry-specific digital infrastructure, mobile robots, and supporting products for industrial digital transformation.
Huang Shijun is a senior strategic planner at ZTE Corporation, with research interests encompassing machine vision, artificial intelligence, computer vision, and deep learning.
Xia Siyu (xsy@seu.edu.cn) received his BE and MS degrees in automation engineering from Nanjing University of Aeronautics and Astronautics, China in 2000 and 2003, respectively, and the PhD degree in pattern recognition and intelligence systems from Southeast University, China in 2006. He is currently an associate professor with the School of Automation, Southeast University. His research interests include object detection, applied machine learning, social media analysis, and intelligent vision systems. He was a recipient of the Science Research Famous Achievement Award from the Higher Institution of China in 2015. He has served as a reviewer for many journals including IEEE T-PAMI, T-IP, T-SMCB, T-IFS, T-MM, and Neurocomputing. He received the Outstanding Reviewer Award for Neurocomputing in 2016. He has also served on the PC/SPC for conferences including CVPR, AAAI, ACM MM, and IJCAI. He is a member of the ACM and IEEE.
Supported by:
ZTE Industry?University?Institute Cooperation Funds(HC?CN?20221107001)

Abstract

Abstract:

The detection of steel surface anomalies has become an industrial challenge due to variations in production equipment, processes, and steel characteristics. To alleviate the problem, this paper proposes a detection and localization method combining 3D depth and 2D RGB features. The framework comprises three stages: defect classification, defect location, and warpage judgment. The first stage uses a data-efficient image Transformer model, the second stage utilizes reverse knowledge distillation, and the third stage performs feature fusion using 3D depth and 2D RGB features. Experimental results show that the proposed algorithm achieves relatively high accuracy and feasibility, and can be effectively used in industrial scenarios.

Key words: anomaly detection, anomaly localization, feature fusion, reverse distillation

Zheng Wangguandong, Lu Ping, Deng Fangwei, Huang Shijun, Xia Siyu. Steel Surface Anomaly Detection Using 3D Depth and 2D RGB Features[J]. ZTE Communications, 2026, 24(1): 81-87.

Figures/Tables 11

Figure 1 Pipeline of our proposed method

Figure 2 Distillation token in transformer model

Figure 3 Difference between traditional knowledge distillation and reverse knowledge distillation

Figure 4 Reverse knowledge distillation network architecture

Figure 5 Aggregation-based feature fusion

Figure 6 Line array cameras capture real scenes and schematic scenes

Table 1 Number of testset and precision of aluminum, iron, and stainless steel samples

Defect Category	Aluminum	Iron	Stainless Steel
Abrasions	8/100%	8/100%	30/96.7%
Scratches	212/97.17%	101/99.1%	18/100%
Holes	28/100%	12/100%	8/100%
Stripes	8/100%	8/100%	8/100%
Flowers	26/100%	8/100%	8/100%

Figure 7 Training loss and validation accuracy of aluminium, iron, and stainless steel samples, where the red curve means training loss and the blue means validation accuracy

Table 2. Pixel-level AUROC and image-level AUROC of aluminum, iron, and stainless steel samples

Figure 8 Classification results of aluminum, iron and stainless steel on five types of defects

Steel Category	Defect Category	Pixel-level AUROC	Image-level AUROC
Aluminum	Abrasions	97.3%	96.7%
	Holes	99.8%	100%
	Scratches	97.5%	100%
Iron	Abrasions	98.1%	100%
	Holes	97.8%	100%
	Scratches	96.3%	100%
Stainless steel	Abrasions	96.4%	100%
	Holes	99.2%	95.4%
	Scratches	85.8%	98.1%

Figure 9 Defect location results by heatmaps

References 16

[1]	Defard T, Setkov A, Loesch A, et al. PaDiM: a patch distribution modeling framework for anomaly detection and localization [C]//ICPR International Workshops and Challenges. ICPR, 2020: 475–489. DOI: 10.1007/978-3-030-68799-1_35
[2]	Tao X, Gong X, Zhang X, et al. Deep learning for unsupervised anomaly localization in industrial images: a survey [J]. IEEE Transactions on instrumentation and measurement, 2022, 71(1): 1–21. DOI: 10.1109/TIM.2022.3196436
[3]	He Y, Song K, Meng Q, et al. An end-to-end steel surface defect detection approach via fusing multiple hierarchical features [J]. IEEE transactions on instrumentation and measurement, 2020, 69(4): 1493–1504. DOI: 10.1109/TIM.2019.2915404
[4]	Zhao J, Liu K, Wang W, et al. Adaptive fuzzy clustering based anomaly data detection in energy system of steel industry [J]. Information sciences, 2014, 259: 335–345. DOI: 10.1016/j.ins.2013.05.018
[5]	Wen X, Zhao W, Yu Z, et al. A novel anomaly detection method for strip steel based on multi-scale knowledge distillation and feature information banks network [J]. Coatings, 2023, 13(7): 1171. DOI: 10.3390/coatings13071171
[6]	Yasuno T, Fujii J, Fukami S. One-class steel detector using patch GAN discriminator for visualising anomalous feature map [PP/OL]. arXiv (2021-06-30) [2025-08-12].
[7]	Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16×16 words: transformers for image recognition at scale [PP/OL]. arXiv (2021-07-03) [2025-08-12].
[8]	Touvron H, Cord M, Douze M, et al. Training data-efficient image transformers & distillation through attention [PP/OL]. arXiv [2025-08-12].
[9]	Salehi M, Sadjadi N, Baselizadeh S, et al. Multiresolution knowledge distillation for anomaly detection [C]//Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2021: 14897–14907. DOI: 10.1109/cvpr46437.2021.01466
[10]	Deng H Q, Li X Y. Anomaly detection via reverse distillation from one-class embedding [C]//Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2022: 9727–9736. DOI: 10.1109/cvpr52688.2022.00951
[11]	Zhu J G, Tang S X, Chen D P, et al. Complementary relation contrastive distillation [C]//Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2021: 9256–9265. DOI: 10.1109/cvpr46437.2021.00914
[12]	Wang Y K, Huang W B, Sun F C, et al. Deep multimodal fusion by channel exchanging [PP/OL]. arXiv [2025-08-12].
[13]	He K, Zhang X, Ren S, et al. Deep residual learning for image recognition [C]//Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2016: 770–778. DOI: 10.1109/CVPR.2016.90
[14]	Zagoruyko S, Komodakis N. Wide residual networks [PP/OL]. arXiv [2025-08-12].
[15]	Roth K, Pemula L, Zepeda J, et al. Towards total recall in industrial anomaly detection [C]//Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2022: 14298–14308. DOI: 10.1109/CVPR52688.2022.01392
[16]	Kingma D P, Ba J. Adam: a method for stochastic optimization [PP/OL]. arXiv (2014-12-22) [2025-08-12].

Steel Surface Anomaly Detection Using 3D Depth and 2D RGB Features

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 11

References 16

Related Articles 3

Recommended Articles

Metrics

[1]	ZHANG Qixun, HAN Jing, CHENG Li, ZHANG Baisheng, GONG Zican. Approach to Anomaly Detection in Microservice System with Multi- Source Data Streams [J]. ZTE Communications, 2022, 20(3): 85-92.
[2]	HAN Jing, JIA Tong, WU Yifan, HOU Chuanjia, LI Ying. Feedback‑Aware Anomaly Detection Through Logs for Large‑Scale Software Systems [J]. ZTE Communications, 2021, 19(3): 88-94.
[3]	LIU Jianwei, YUAN Yifei, HAN Jing. A Case Study on Intelligent Operation System for Wireless Networks [J]. ZTE Communications, 2019, 17(4): 19-26.