ZTE Communications ›› 2023, Vol. 21 ›› Issue (1): 46-54.DOI: 10.12142/ZTECOM.202301006
-
收稿日期:2023-02-11出版日期:2023-03-25发布日期:2024-03-15
Secure Federated Learning over Wireless Communication Networks with Model Compression
DING Yahao1, SHIKH‑BAHAEI Mohammad1, YANG Zhaohui2, HUANG Chongwen2(
), YUAN Weijie3
- 1.King’s College London, London WC2R 2LS, U. K.
2.Zhejiang University, Hangzhou 310058, China
3.Southern University of Science and Technology, Shenzhen 518055, China
-
Received:2023-02-11Online:2023-03-25Published:2024-03-15 -
About author:DING Yahao received her master's degree in communications and signal processing from Imperial College London, U.K. in 2020. She is currently pursuing her PhD degree in information and communication engineering with King's College London, U.K. Her current research interests include federated learning, security, and UAV swarms.
Mohammad SHIKH-BAHAEI received his BSc degree from the University of Tehran, Iran in 1992, MSc degree from the Sharif University of Technology, Iran in 1994, and PhD degree from King's College London, U.K. in 2000. He has worked for two start-up companies and for National Semiconductor Corporation, USA (now part of Texas Instruments Inc.). In 2002, he joined King's College London, where he is currently a full professor. Since then, he has authored numerous journals and conference papers and worked as an expert consultant to a number of international high-tech companies and legal firms. His research interests are secure communications and connected intelligence, full-duplex and cognitive dense networks, visual data communications over the IoT, applications of wireless communications in healthcare, and communication protocols for autonomous vehicle/drone networks. He has been the founder and the chair of the Wireless Advanced (formerly SPWC) Annual International Conference from 2003 to 2018.
YANG Zhaohui received his PhD degree from Southeast University, China in 2018. From 2018 to 2020, he was a postdoctoral research associate with the Center for Telecommunications Research, Department of Informatics, King's College London, U.K. From 2020 to 2022, he was a research fellow with the Department of Electronic and Electrical Engineering, University College London, U.K. He is currently a young professor with the College of Information Science and Electronic Engineering, Zhejiang Key Laboratory of Information Processing Communication and Networking, Zhejiang University, China, and also a research scientist with Zhejiang Laboratory. His research interests include joint communication, sensing and computation, federated learning, and semantic communications. He is an associate editor for IEEE Communications Letters, IET Communications, and EURASIP Journal on Wireless Communications and Networking. He was the guest editor of several journals, including JSAC, WCM and CM. He was the co-chair for international workshops with more than ten times, including ICC, GLOBECOM, WCNC, PIMRC and INFOCOM.
HUANG Chongwen (chongwenhuang@zju.edu.cn) received his BSc degree from the Binhai College, Nankai University, China in 2010, and MSc degree from the University of Electronic Science and Technology of China (UESTC), China in 2013. He has been joining the Institute of Electronics, Chinese Academy of Sciences (IECAS) as a research engineer, since July 2013. Since September 2015, he has been starting his PhD journey with the Singapore University of Technology and Design (SUTD), Singapore and CentraleSupélec University, Paris, France under the supervision of Prof. Chau YUEN and Prof. Mérouane DEBBAH. From October 2019 to September 2020, he was a post-doctoral researcher at SUTD. Since September 2020, he has been joining Zhejiang University as a Tenure-Track Young Professor. His main research interests include holographic MIMO surface/reconfigurable intelligent surface, B5G/6G wireless communications, mmWave/THz communications, and deep learning technologies for wireless communications. He was a recipient of the IEEE Marconi Prize Paper Award in Wireless Communications in 2021. He was also a recipient of the Singapore Government PhD Scholarship and received PHC Merlion PhD Grant (2016–2019) for studying in CentraleSupélec, France. He has been serving as an editor of IEEE Communications Letter, Signal Processing (Elsevier), EURASIP Journal on Wireless Communications and Networking, and Physical Communication since 2021. In addition, he has served as the chair of several wireless communications flagship conferences, including the session chair of 2021 IEEE WCNC, 2021 IEEE VTC-Fall, and the symposium chair of IEEE WCSP 2021.
YUAN Weijie received his BE degree from the Beijing Institute of Technology, China in 2013, and PhD degree from the University of Technology Sydney, Australia in 2019. In 2016, he was a visiting PhD student with the Institute of Telecommunications, Vienna University of Technology, Austria. He was a research assistant with the University of Sydney, Australia, a visiting associate fellow with the University of Wollongong, Australia and a visiting fellow with the University of Southampton, U.K. from 2017 to 2019. From 2019 to 2021, he was a research associate with the University of New South Wales, Australia. He is currently an assistant professor with the Department of Electrical and Electronic Engineering, Southern University of Science and Technology, China. He was a recipient of the Best PhD Thesis Award from the Chinese Institute of Electronics and an Exemplary Reviewer from IEEE TCOM/WCL. He currently serves as an associate editor of IEEE Communications Letters, an associate editor and an award committee member of EURASIP Journal on Advances in Signal Processing. He has led the guest editorial teams for three special issues in IEEE Communications Magazine, IEEE Transactions on Green Communications and Networking, and China Communications. He was an organizer/the chair of several workshops and special sessions on orthogonal time frequency space and integrated sensing and communication in flagship IEEE and ACM conferences, including IEEE ICC, IEEE/CIC ICCC, IEEE SPAWC, IEEE VTC, IEEE WCNC, IEEE ICASSP, and ACM MobiCom. He is the founding chair of the IEEE ComSoc Special Interest Group on Orthogonal Time Frequency Space.
引用本文
. [J]. ZTE Communications, 2023, 21(1): 46-54.
DING Yahao, SHIKH‑BAHAEI Mohammad, YANG Zhaohui, HUANG Chongwen, YUAN Weijie. Secure Federated Learning over Wireless Communication Networks with Model Compression[J]. ZTE Communications, 2023, 21(1): 46-54.
Figure 1 Architecture of FL algorithm with one eavesdropper in wireless networks
Figure 1 Architecture of FL algorithm with one eavesdropper in wireless networks
Figure 2 Illustration of the differential privacy (DP) method to protect the privacy of federated learning (FL)
Figure 2 Illustration of the differential privacy (DP) method to protect the privacy of federated learning (FL)
Figure 3 An example of proposed weight compression
Figure 3 An example of proposed weight compression
Table 1 Simulation Parameters
| Description | Parameter | Value |
|---|---|---|
| Total bandwidth of uplink | B | 20 MHz |
| Bandwidth of each RB | B0 | 3.33 MHz |
| Noise power spectral density | N0 | -174 dBm/MHz |
| Total number of training samples for user | Ki | [10, 20, 15, 25, 10] |
| Gradient compression ratio of user | αi | |
| Number of gradients for each user | M | 9 |
| Delay requirement of uplink | τ | 2 s |
| Distance between user and BS | d | 30 m |
| Number of RBs | Q | 6 |
| Transmit power of user | Pi | 0.001–0.012 W |
Table 1 Simulation Parameters
| Description | Parameter | Value |
|---|---|---|
| Total bandwidth of uplink | B | 20 MHz |
| Bandwidth of each RB | B0 | 3.33 MHz |
| Noise power spectral density | N0 | -174 dBm/MHz |
| Total number of training samples for user | Ki | [10, 20, 15, 25, 10] |
| Gradient compression ratio of user | αi | |
| Number of gradients for each user | M | 9 |
| Delay requirement of uplink | τ | 2 s |
| Distance between user and BS | d | 30 m |
| Number of RBs | Q | 6 |
| Transmit power of user | Pi | 0.001–0.012 W |
Figure 4 Objective function as user power and resource block (RB) allocation varies
Figure 4 Objective function as user power and resource block (RB) allocation varies
| 1 | YANG Q, LIU Y, CHEN T J, et al. Federated machine learning: concept and applications [J]. ACM transactions on intelligent systems and technology, 2019, 10(2): 1–19. DOI: 10.1145/3298981 |
| 2 | JOCHEMS A, DEIST T M, VAN SOEST J, et al. Distributed learning: developing a predictive model based on data from multiple hospitals without data leaving the hospital—a real life proof of concept [J]. Radiotherapy and oncology, 2016, 121(3): 459–467. DOI: 10.1016/j.radonc.2016.10.002 |
| 3 | BONAWITZ K, EICHNER H, GRIESKAMP W, et al. Towards federated learning at scale: system design [C]//Conference on machine learning and systems. MLSys, 2019: 374–388 |
| 4 | SHOKRI R, STRONATI M, SONG C Z, et al. Membership inference attacks against machine learning models [C]//IEEE Symposium on Security and Privacy (SP). IEEE, 2017: 3–18. DOI: 10.1109/SP.2017.41 |
| 5 | HITAJ B, ATENIESE G, PEREZ-CRUZ F. Deep models under the GAN: information leakage from collaborative deep learning [C]//ACM SIGSAC Conference on Computer and Communications Security. ACM, 2017: 603–618. DOI: 10.1145/3133956.3134012 |
| 6 | WANG Z B, SONG M K, ZHANG Z F, et al. Beyond inferring class representatives: user-level privacy leakage from federated learning [C]//IEEE Conference on Computer Communications. IEEE, 2019: 2512–2520. DOI: 10.1109/INFOCOM.2019.8737416 |
| 7 | PHONG L T, AONO Y, HAYASHI T, et al. Privacy-preserving deep learning via additively homomorphic encryption [J]. IEEE transactions on information forensics and security, 2017, 13(5): 1333–1345. DOI: 10.1109/TIFS.2017.2787987 |
| 8 | ZHU L G, LIU Z J, HAN S. Deep leakage from gradients [C]//33rd Conference on Neural Information Processing Systems. NeurIPS, 2019: 8389 |
| 9 | ZHAO B, MOPURI K R, BILEN H. iDLG: improved deep leakage from gradients [EB/OL]. [2020-01-08]. |
| 10 | YIN H X, MALLYA A, VAHDAT A, et al. See through gradients: image batch recovery via GradInversion [C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2021: 16332–16341. DOI: 10.1109/CVPR46437.2021.01607 |
| 11 | CHEN S, JIA R X, QI G J. Improved techniques for model inversion attack [EB/OL]. [2021-08-19]. |
| 12 | JERE M S, FARNAN T, KOUSHANFAR F. A taxonomy of attacks on federated learning [J]. IEEE security privacy, 2021, 19(2): 20–28. DOI: 10.1109/MSEC.2020.3039941 |
| 13 | GEIPING J, BAUERMEISTER H, DRÖGE H, et al. Inverting gradients: how easy is it to break privacy in federated learning? [C]//34th International Conference on Neural Information Processing Systems. NeurIPS, 2020: 16937–16947 |
| 14 | ZHANG C L, LI S Y, XIA J Z, et al. BatchCrypt: efficient homomorphic encryption for cross-silo federated learning [C]//USENIX Conference on Usenix Annual Technical Conference. ACM, 2020: 493–506. DOI: 10.5555/3489146.3489179 |
| 15 | CHENG K W, FAN T, JIN Y L, et al. SecureBoost: a lossless federated learning framework [J]. IEEE intelligent systems, 2021, 36(6): 87–98. DOI: 10.1109/MIS.2021.3082561 |
| 16 | MOHASSEL P, ZHANG Y P. Secureml: a system for scalable privacy-preserving machine learning [C]//IEEE Symposium on Security and Privacy (SP). IEEE, 2017: 19–38. DOI: 10.1109/SP.2017.12 |
| 17 | BONAWITZ K, IVANOV V, KREUTER B, et al. Practical secure aggregation for privacy-preserving machine learning [C]//ACM SIGSAC Conference on Computer and Communications Security. ACM, 2017: 1175–1191. DOI: 10.1145/3133956.3133982 |
| 18 | GEYER R C, KLEIN T, NABI M. Differentially private federated learning: a client level perspective [EB/OL]. [2022-03-01]. |
| 19 | MCMAHAN H B, RAMAGE D, TALWAR K, et al. Learning differentially private recurrent language models [EB/OL]. [2022-02-24]. |
| 20 | WEI K, LI J, DING M, et al. Federated learning with differential privacy: algorithms and performance analysis [J]. IEEE transactions on information forensics and security, 2020, 15: 3454–3469. DOI: 10.1109/TIFS.2020.2988575 |
| 21 | WEI W Q, LIU L, LOPER M, et al. A framework for evaluating gradient leakage attacks in federated learning [EB/OL]. [2021-04-23]. |
| 22 | SUN J W, LI A, WANG B H, et al. Soteria: provable defense against privacy leakage in federated learning from representation perspective [C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2021: 9307–9315. DOI: 10.1109/CVPR46437.2021.00919 |
| 23 | SCHELIGA D, MÄDER P, SEELAND M. Precode—a generic model extension to prevent deep gradient leakage [C]//IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). IEEE, 2022: 3605–3614. DOI: 10.1109/WACV51458.2022.00366 |
| 24 | WANG J X, GUO S, XIE X, et al. Protect privacy from gradient leakage attack in federated learning [C]//IEEE Conference on Computer Communications. IEEE, 2022: 580–589. DOI: 10.1109/INFOCOM48880.2022.9796841 |
| 25 | KONEČNÝ J, MCMAHAN H B, RAMAGE D, et al. Federated optimization: distributed machine learning for on-device intelligence [EB/OL]. [2021-10-08]. |
| 26 | HATAMIZADEH A, YIN H X, MOLCHANOV P, et al. Do gradient inversion attacks make federated learning unsafe? [EB/OL]. [2022-02-14]. |
| 27 | TAVANGARAN N, CHEN M Z, YANG Z H, et al. On differential privacy for federated learning in wireless systems with multiple base stations [EB/OL]. [2022-08-25]. |
| 28 | CHEN M Z, YANG Z H, SAAD W, et al. A joint learning and communications framework for federated learning over wireless networks [J]. IEEE transactions on wireless communications, 2021, 20(1): 269–283. DOI: 10.1109/TWC.2020.3024629 |
| No related articles found! |
| 阅读次数 | ||||||
|
全文 |
|
|||||
|
摘要 |
|
|||||