ACTap: Integrating Attention and Convolution for Network Modality Recognition

doi:10.12142/ZTECOM.202602009

ZTE Communications ›› 2026, Vol. 24 ›› Issue (2): 71-82.DOI: 10.12142/ZTECOM.202602009

• Industry-Academia Co-Research • Previous Articles Next Articles

ACTap: Integrating Attention and Convolution for Network Modality Recognition

Ling Zihan¹, Zhang Tianwei¹, Ning Yuwei¹, Cao Yang¹(), Shen Can²

^1.Huazhong University of Science and Technology, Wuhan 430074, China
^2.ZTE Corporation, Shenzhen 518057, China

Received:2025-08-14 Online:2026-06-25 Published:2026-06-16
About author:Ling Zihan received his BE degree from the School of Microelectronics and Communication Engineering, Chongqing University, China in 2023. He is currently pursuing the MS degree at the School of Electronic Information and Communications, Huazhong University of Science and Technology, China. His current research interests include real-time VR video transmission, network modal analysis, and adaptive bitrate algorithms.
Zhang Tianwei received his MS degree in wireless communication systems from the University of Sheffield, UK in 2020. He is currently pursuing the PhD degree at the School of Electronic Information and Communications, Huazhong University of Science and Technology, China. His current research interests include real-time video transmission, video processing, and adaptive streaming.
Ning Yuwei is a third-year undergraduate student at Huazhong University of Science and Technology, China, supervised by Prof. Cao Yang. His research interests include deep learning, time series processing, and computer vision.
Cao Yang (ycao@hust.edu.cn) is currently an associate professor at the School of Electronic Information and Communications, Huazhong University of Science and Technology, China. From 2011 to 2013, he was a visiting scholar at the School of Electrical, Computer, and Energy Engineering, Arizona State University, USA. His research interests include immersive video transmission and edge/distributed computing. He has coauthored 50 papers in refereed IEEE journals and conferences. He received the CHINACOM Best Paper Award in 2010 and the Microsoft Research Fellowship in 2011.
Shen Can received his PhD degree in engineering in 1997 and has since been engaged in the research and development of audio and video technologies at ZTE Corporation. He has received the Second Prize of the National Science and Technology Progress Award, published 20 papers, and holds over 70 patents.
Supported by:
This work was supported in part by ZTE Industry?University?Institute Cooperation Funds under Grant No. IA20230728008 and the National Natural Science Foundation of China (NSFC) under Grant No. 62271224.

Abstract

Abstract:

With the sustained growth of live video streaming, the demand for high-quality video services for mobile users across diverse network environments is increasing rapidly. In this paper, we define the network fluctuation characteristics in different environments as network modality. To comprehensively investigate network modality across different environments, we construct a network modality dataset by collecting multi-dimensional network metrics from various real-world scenarios and suggest that network modality exhibits separability. Therefore, network modality recognition, which aims to distinguish the scenarios where a user is located based on network modality sequences, is feasible and can be formulated as a multivariate time series (MTS) classification problem. To address this problem, we propose a novel neural network (NN)-based classification model called ACTap. Specifically, the model first integrates a two-stage attention (TSA) mechanism and a convolutional neural network (CNN) to extract features from network modality sequences. Then, it filters out noisy feature representations to learn discriminative class prototypes, and finally recognizes network modality based on the distance between their feature representations and class prototypes. Experimental results validate the separability of network modality and show that ACTap outperforms four benchmark models in terms of classification accuracy on the network modality dataset.

Key words: network modality, multivariate time series classification, feature fusion, class prototype learning

Ling Zihan, Zhang Tianwei, Ning Yuwei, Cao Yang, Shen Can. ACTap: Integrating Attention and Convolution for Network Modality Recognition[J]. ZTE Communications, 2026, 24(2): 71-82.

Figures/Tables 14

Table 1 Comparison of network datasets

Dataset	Sampling Time Interval/s	Multi‑Dimensional Modality Information	Multiple Scenes	Full Pipe	Protocol
Ours	0.1	√	√	√	UDP/TCP
Raca et al.^[4]	1	√	√	×	TCP
Yang et al.^[5]	unknown	×	√	×	UDP/TCP
Van et al.^[6]	1	×	√	×	TCP
Pan et al.^[7]	1	√	×	×	UDP/TCP
Kousias et al.^[8]	millisecond-level	√	√	×	TCP

Figure 1 Bandwidth samples in six scenarios

Table 2 Summary of notations

Symbol	Description
$X n$	The network modality sequence
$y n ∈ Ω$	The corresponding class label of the network modality sequence
$L$	The length of the network modality sequence
$F n$	The feature representation of the network modality sequence
$f l e n$	The length of the feature representation
$s d, i$	The $i$ -th segment in dimension $d$
$s e g l e n$	The length of segment
$h d, i$	The $i$ -th feature vector in dimension $d$
$d m o d e l$	The length of the feature vector
$H$	The input of the TSA layer
$H t i m e, H d i m$	The outputs of the cross-time stage and cross-dimension stage, respectively
$F T S A, F C N N$	The extracted features of the TSA layer and CNN layer, respectively
$P k, P ˜ k$	The set of all representations and the set of selected representations, respectively
$W k$	The attention weight for class $k$
$c k$	The prototype of class $k$
$D (x, c k)$	The distance between network modality sequence $x$ and class prototype $c k$
$p (y = k \| x)$	The probability of network modality sequence $x$ belonging to class $k$

Table 2 Summary of notations

Symbol	Description
$X n$	The network modality sequence
$y n ∈ Ω$	The corresponding class label of the network modality sequence
$L$	The length of the network modality sequence
$F n$	The feature representation of the network modality sequence
$f l e n$	The length of the feature representation
$s d, i$	The $i$ -th segment in dimension $d$
$s e g l e n$	The length of segment
$h d, i$	The $i$ -th feature vector in dimension $d$
$d m o d e l$	The length of the feature vector
$H$	The input of the TSA layer
$H t i m e, H d i m$	The outputs of the cross-time stage and cross-dimension stage, respectively
$F T S A, F C N N$	The extracted features of the TSA layer and CNN layer, respectively
$P k, P ˜ k$	The set of all representations and the set of selected representations, respectively
$W k$	The attention weight for class $k$
$c k$	The prototype of class $k$
$D (x, c k)$	The distance between network modality sequence $x$ and class prototype $c k$
$p (y = k \| x)$	The probability of network modality sequence $x$ belonging to class $k$

Figure 2 Overall architecture of ACTap

Figure 3 Partition of network modality sequence

Figure 4 The measurement setup consists of 1) the CPE, 2) a portable power source, and 3) the mobile device with the HoloWan software

Table 3 Accuracy performance of models

Model	Accuracy
ACTap	0.902 3
TapNet	0.671 6
CNNTap	0.850 0
AttentionTap	0.813 3
ACMLP	0.847 9

Figure 5 Performance comparison under different lengths of network modality sequence

Figure 6 Performance comparison under different numbers of clusters

Figure 7 Performance comparison of feature selection methods

Figure 8 Performance comparison under different sizes of the training set

Figure 9 Partitioning the network modality sequence into overlapping segments

Figure 10 Performance comparison under different overlapping levels

Figure 11 Classification visualization of ACTap

References 34

[1]	Carlucci G, De Cicco L, Holmer S, et al. Analysis and design of the google congestion control for web real-time communication (WebRTC) [C]//Proceedings of the 7th International Conference on Multimedia Systems. ACM, 2016: 1–12. DOI: 10.1145/2910017.2910605
[2]	Mao H Z, Netravali R, Alizadeh M. Neural adaptive video streaming with Pensieve [C]//Proceedings of the Conference of the ACM Special Interest Group on Data Communication. ACM, 2017: 197–210. DOI: 10.1145/3098822.3098843
[3]	Zhang H H, Zhou A F, Lu J M, et al. OnRL: improving mobile video telephony via online reinforcement learning [C]//Proceedings of the 26th Annual International Conference on Mobile Computing and Networking. ACM, 2020: 1–14. DOI: 10.1145/3372224.3419186
[4]	Raca D, Leahy D, Sreenan C J, et al. Beyond throughput, the next generation: a 5G dataset with channel and context metrics [C]//Proceedings of the 11th ACM Multimedia Systems Conference. ACM, 2020: 303–308. DOI: 10.1145/3339825.3394938
[5]	Yang L, Yuan M X, Wang W, et al. Apps on the move: a fine-grained analysis of usage behavior of mobile apps [C]//Proceedings of the 35th Annual IEEE International Conference on Computer Communications. IEEE, 2016: 1–9. DOI: 10.1109/INFOCOM.2016.7524464
[6]	van der Hooft J, Petrangeli S, Wauters T, et al. HTTP/2-based adaptive streaming of HEVC video over 4G/LTE networks [J]. IEEE communications letters, 2016, 20(11): 2177–2180. DOI: 10.1109/LCOMM.2016.2601087
[7]	Pan Y Y, Li R H, Xu C R. The first 5G-LTE comparative study in extreme mobility [J]. Proceedings of the ACM on measurement and analysis of computing systems, 2022, 6(1): 1–22. DOI: 10.1145/3508040
[8]	Kousias K, Rajiullah M, Caso G, et al. A large-scale dataset of 4G, NB-IoT, and 5G non-standalone network measurements [J]. IEEE communications magazine, 2024, 62(5): 44–49. DOI: 10.1109/MCOM.011.2200707
[9]	Lu J G, Zheng Q F. Ultra-lightweight face animation method for ultra-low bitrate video conferencing [J]. ZTE communications, 2023, 21(1): 64–71. DOI: 10.12142/ZTECOM.202301008
[10]	Xie L, Zhang X G, Huang C, et al. Markov based rate adaption approach for live streaming over HTTP/2 [J]. ZTE communications, 2018, 16(2): 37–41. DOI: 10.3969/j.issn.1673-5188.2018.02.007
[11]	HoloWan APP [EB/OL]. [2024-08-01].
[12]	ZHANG Y H, YAN J C. Crossformer: transformer utilizing cross-dimension dependency for multivariate time series forecasting [C/OL]//The Eleventh International Conference on Learning Representations. ICLR, 2022.
[13]	Zhang X C, Gao Y F, Lin J, et al. TapNet: multivariate time series classification with attentional prototypical network [J]. Proceedings of the AAAI conference on artificial intelligence, 2020, 34(4): 6845–6852. DOI: 10.1609/aaai.v34i04.6165
[14]	Li G Z, Choi B, Xu J L, et al. ShapeNet: a shapelet-neural network approach for multivariate time series classification [J]. Proceedings of the AAAI conference on artificial intelligence, 2021, 35(9): 8375–8383. DOI: 10.1609/aaai.v35i9.17018
[15]	Gao G, Gao Q T, Yang X, et al. A reinforcement learning-informed pattern mining framework for multivariate time series classification [C]//31st International Joint Conference on Artificial Intelligence. IJCAI, 2022: 2994–3000. DOI: 10.24963/ijcai.2022/415
[16]	Khan M, Wang H Z, Ngueilbaye A, et al. End-to-end multivariate time series classification via hybrid deep learning architectures [J]. Personal and ubiquitous computing, 2023, 27(2): 177–191. DOI: 10.1007/s00779-020-01447-7
[17]	Zerveas G, Jayaraman S, Patel D, et al. A transformer-based framework for multivariate time series representation learning [C]//Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. ACM, 2021: 2114–2124. DOI: 10.1145/3447548.3467401
[18]	Zuo R D, Li G Z, Choi B, et al. SVP-T: a shape-level variable-position transformer for multivariate time series classification [J]. Proceedings of the AAAI conference on artificial intelligence, 2023, 37(9): 11497–11505. DOI: 10.1609/aaai.v37i9.26359
[19]	Gao N Z, Yu Y F, Hua X H. A content-aware bitrate selection method using multi-step prediction for 360-degree video streaming [J]. ZTE communications, 2022, 20(4): 96–109. DOI: 10.12142/ZTECOM.202204012
[20]	Li J X, Xu Y T, Cao Y, et al. Utility-driven joint caching and bitrate allocation for real-time immersive videos [J]. IEEE journal of selected topics in signal processing, 2023, 17(5): 1106–1118. DOI: 10.1109/JSTSP.2023.3295597
[21]	Xu Y T, Du J H, Wang J H, et al. Panonut360: a head and eye tracking dataset for panoramic video [C]//Proceedings of the 15th ACM Multimedia Systems Conference. ACM, 2024: 319–325. DOI: 10.1145/3625468.3652176
[22]	Du D Z, Su B, Wei Z W. Preformer: predictive transformer with multi-scale segment-wise correlations for long-term time series forecasting [C]//2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023: 1–5. DOI: 10.1109/ICASSP49357.2023.10096881
[23]	Zhou H Y, Zhang S H, Peng J Q, et al. Informer: beyond efficient transformer for long sequence time-series forecasting [J]. Proceedings of the AAAI conference on artificial intelligence, 2021, 35(12): 11106–11115. DOI: 10.1609/aaai.v35i12.17325
[24]	Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16×16 words: transformers for image recognition at scale [PP/OL]. arXiv (2021-06-03) [2024-08-01].
[25]	Vaswani A, Shazeer N, Parmar N, et al. 2017. Attention is all you need [C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. NIPS, 2017: 6000–6010. DOI:10.48550/arXiv.1706.03762
[26]	Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift [C]//International conference on machine learning. PMLR, 2015: 448-456. DOI: 10.48550/arXiv.1502.03167
[27]	Xu B, Wang N Y, Chen T Q, et al. Empirical evaluation of rectified activations in convolutional network [PP/OL]. V2. arXiv (2015-11-27). [2024-06-07].
[28]	MacQueen J. Some methods for classification and analysis of multivariate observations [C]//Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Statistics. University of California Press, 1967: 281–297
[29]	Kingma D P, Ba J. Adam: a method for stochastic optimization [PP/OL]. V9. arXiv (2017-01-30) [2024-06-07].
[30]	Aliyun [EB/OL]. [2024-05-19].
[31]	Cardwell N, Cheng Y, Gunn C S, et al. BBR: congestion-based congestion control [J]. Communications of the ACM, 2017, 60(2): 58–66. DOI: 10.1145/3009824
[32]	Pytorch [EB/OL]. [2024-05-19].
[33]	Bagnall A, Dau H A, Lines J, et al. The UEA multivariate time series classification archive, 2018 [PP/OL]. V1. arXiv (2018-10-31)[2024-04-21].
[34]	van der Maaten L, Hinton G. Visualizing data using t-SNE [J/OL]. Journal of machine learning research, 2008, 9(86): 2579-2605.

ACTap: Integrating Attention and Convolution for Network Modality Recognition

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 14

References 34

Related Articles 1

Recommended Articles

Metrics