ZTE Communications ›› 2022, Vol. 20 ›› Issue (3): 43-53.DOI: 10.12142/ZTECOM.202203006
• Special Topic • Previous Articles Next Articles
TANG Bo1, ZHANG Chengming1, WANG Kewen1, GAO Zhengguang2, HAN Bingtao2()
Received:
2022-06-18
Online:
2022-09-13
Published:
2022-09-14
About author:
TANG Bo received his master's degree from Northwestern Polytechnical University, China in 2005. Currently, as the system architect of ZTE Corporation, he is mainly responsible for federated learning and AI security solutions. His current research interests include container and container cloud, heterogeneous resource scheduling, and AI security and trustworthy AI. He holds several patents in the above research areas.|ZHANG Chengming received his BS degree from Yangzhou University, China in 2011, and ME degree from Nanjing University of Posts and Telecommunications, China in 2015. He is a senior R&D engineer of ZTE Corporation. His current research interests include AI platform, container cloud, resource scheduling, federated learning, and AI security.|WANG Kewen received his BS degree from China University of Mining and Technology, China in 2015, and the ME degree from Beijing Institute of Technology, China in 2017. He is an AI senior algorithm engineer of ZTE Corporation. His current research interests include AI Systems and AI security, such as federated learning, adversarial attack, and defense in deep learning.|GAO Zhengguang received his PhD degree from Beijing University of Posts and Telecommunications, China in 2020. He was a visiting PhD student in High Performance Networks Group, University of Bristol, UK from 2018 to 2019. After graduation, he was selected for “LAN JIAN” program of ZTE Corporation as an algorithm researcher. His current research interests include 5G/6G communication technologies, mobile networks, and machine learning for future communications.|HAN Bingtao (TANG Bo, ZHANG Chengming, WANG Kewen, GAO Zhengguang, HAN Bingtao. Neursafe-FL: A Reliable, Efficient, Easy-to- Use Federated Learning Framework[J]. ZTE Communications, 2022, 20(3): 43-53.
Add to citation manager EndNote|Ris|BibTeX
URL: http://zte.magtechjournal.com/EN/10.12142/ZTECOM.202203006
Concerns | Features | TFF | PySyft | FedML | FATE | PaddleFL | Neursafe-FL |
---|---|---|---|---|---|---|---|
Supported running mode | Standalone | √ | √ | √ | √ | √ | √ |
Cross-device | × | × | √ | × | × | √ | |
Cross-silo | × | × | √ | √ | √ | √ | |
Aggregation algorithms | IID (FedAvg, etc.) | √ | √ | √ | √ | √ | √ |
Non-IID (FedProx, etc.) | × | × | √ | √ | - | √ | |
Supported underlying framework | Tensorflow | √ | √ | × | √ | × | √ |
Pytorch | × | √ | √ | √ | × | √ | |
Privacy protection methods | DP | √ | √ | √ | × | √ | √ |
MPC | × | √ | × | √ | √ | √ | |
HE | × | √ | × | √ | × | × | |
Flexibility | Device management | × | × | × | × | × | √ |
Customization | × | × | √ | × | × | √ |
Table 1 Comparison of open source frameworks
Concerns | Features | TFF | PySyft | FedML | FATE | PaddleFL | Neursafe-FL |
---|---|---|---|---|---|---|---|
Supported running mode | Standalone | √ | √ | √ | √ | √ | √ |
Cross-device | × | × | √ | × | × | √ | |
Cross-silo | × | × | √ | √ | √ | √ | |
Aggregation algorithms | IID (FedAvg, etc.) | √ | √ | √ | √ | √ | √ |
Non-IID (FedProx, etc.) | × | × | √ | √ | - | √ | |
Supported underlying framework | Tensorflow | √ | √ | × | √ | × | √ |
Pytorch | × | √ | √ | √ | × | √ | |
Privacy protection methods | DP | √ | √ | √ | × | √ | √ |
MPC | × | √ | × | √ | √ | √ | |
HE | × | √ | × | √ | × | × | |
Flexibility | Device management | × | × | × | × | × | √ |
Customization | × | × | √ | × | × | √ |
Type | Round Number | Noise_multiplier | Time Cost/s | Accuracy |
---|---|---|---|---|
Without DP | 100 | - | 2 873.22 | 0.914 3 |
50 | - | 1 396.56 | 0.898 4 | |
With DP | 100 | 0.01 | 2 879.36 | 0.912 2 |
100 | 0.005 | 2 882.86 | 0.891 0 | |
50 | 0.01 | 1 390.42 | 0.876 3 |
Table 2 Results of different parameters for differential privacy
Type | Round Number | Noise_multiplier | Time Cost/s | Accuracy |
---|---|---|---|---|
Without DP | 100 | - | 2 873.22 | 0.914 3 |
50 | - | 1 396.56 | 0.898 4 | |
With DP | 100 | 0.01 | 2 879.36 | 0.912 2 |
100 | 0.005 | 2 882.86 | 0.891 0 | |
50 | 0.01 | 1 390.42 | 0.876 3 |
1 | MCMAHAN B, MOORE E, RAMAGE D, et al. Communication-efficient learning of deep networks from decentralized data [C]//International Conference on Artificial Intelligence and Statistics. PMLR, 2017: 1273–1282 |
2 | YANG Q, LIU Y, CHEN T J, et al. Federated machine learning: concept and applications [EB/OL]. [2022-04-01]. |
3 | BONAWITZ K, EICHNER H, GRIESKAMP W, et al. Towards federated learning at scale: system design [EB/OL]. [2022-04-01]. |
4 |
KAIROUZ E B P, MCMAHAN H B. Advances and open problems in federated learning [J]. Foundations and trends in machine learning, 2021, 14(1): 1–21. DOI: 10.1561/2200000083
DOI |
5 |
LI T, SAHU A K, TALWALKAR A, et al. Federated learning: challenges, methods, and future directions [J]. IEEE signal processing magazine, 2020, 37(3): 50–60. DOI: 10.1109/MSP.2020.2975749
DOI |
6 | VOIGT P, VON DEM BUSSCHE A. The EU general data protection regulation (GDPR): a practical guide [M]. Cham: Springer International Publishing, 2017 |
7 |
LONG G D, TAN Y, JIANG J, et al. Federated learning for open banking federated learning [J]. Federated Learning 2020: 240–254. DOI: 10.1007/978-3-030-63076-8_17
DOI |
8 |
XU J, GLICKSBERG B S, SU C, et al. Federated learning for healthcare informatics [J]. Journal of healthcare informatics research, 2021, 5(1): 1–19. DOI: 10.1007/s41666-020-00082-4
DOI |
9 |
JIANG J C, KANTARCI B, OKTUG S, et al. Federated learning in smart city sensing: challenges and opportunities [J]. Sensors, 2020, 20(21): 6230. DOI: 10.3390/s20216230
DOI |
10 | LYU L, YU H, YANG Q. Threats to federated learning: a survey [EB/OL]. [2022-04-01]. |
11 | BAGDASARYAN E, VEIT A, HUA Y, et al. How to backdoor federated learning [C]//International Conference on Artificial Intelligence and Statistics. PMLR, 2020: 2938–2948 |
12 |
WANG Z B, SONG M K, ZHANG Z F, et al. Beyond inferring class representatives: user-level privacy leakage from federated learning [C]//IEEE Conference on Computer Communications. IEEE, 2019: 2512–2520. DOI: 10.1109/INFOCOM.2019.8737416
DOI |
13 |
NASR M, SHOKRI R, HOUMANSADR A. Comprehensive privacy analysis of deep learning: passive and active white-box inference attacks against centralized and federated learning [C]//IEEE Symposium on Security and Privacy. IEEE, 2019: 739–753. DOI: 10.1109/SP.2019.00065
DOI |
14 | ZHAO Y, LI M, LAI L, et al. Federated learning with non-IID [EB/OL]. (2018-06-02) [2022-04-01]. |
15 | LI X, HUANG K, YANG W, et al. On the convergence of FedAvg on non-IID data [EB/OL]. (2020-06-25) [2022-04-01]. |
16 |
ZHANG J L, CHEN J J, WU D, et al. Poisoning attack in federated learning using generative adversarial nets [C]//18th IEEE International Conference on Trust, Security and Privacy in Computing and Communications. IEEE, 2019: 374–380. DOI: 10.1109/TrustCom/BigDataSE.2019.00057
DOI |
17 | FANG M, CAO X, JIA J, et al. Local model poisoning attacks to byzantine-robust federated learning [C]//29th USENIX Security Symposium. USENIX, 2020: 1605–1622 |
18 | REISIZADEH A, TZIOTIS I, HASSANI H, et al. Straggler-resilient federated learning: leveraging the interplay between statistical accuracy and system heterogeneity [EB/OL]. [2022-04-01]. |
19 | DWORK C. Differential privacy: a survey of results [C]//International Conference on Theory and Applications of Models of Computation. TAMC, 2008: 1–19 |
20 | MCMAHAN H B, ANDREW G, ERLINGSSON U, et al. A general approach to adding differential privacy to iterative training procedures [EB/OL]. [2022-04-01]. |
21 | ABADI M, CHU A, GOODFELLOW I, et al. Deep learning with differential privacy [C]//ACM SIGSAC Conference on Computer and Communications Security. ACM, 2016: 308–318 |
22 | DWORK C, ROTH A. The algorithmic foundations of differential privacy [J]. Foundations and trends in theoretical computer science, 2014, 9(3–4): 211–407 |
23 |
BU Z, DONG J, LONG Q, et al. Deep learning with gaussian differential privacy [J]. Harvard data science review, 2020, 23. DOI: 10.1162/99608f92.cfc5dd25
DOI |
24 | GOLDREICH O. Secure multi-party computation [EB/OL]. [2022-04-01]. |
25 | DU W, ATALLAH M J. Secure multi-party computation problems and their applications: a review and open problems [C]//Proceedings of the 2001 Workshop on New Security Paradigms. ACM, 2001: 13–22 |
26 |
MOHASSEL P, RINDAL P. ABY3: a mixed protocol framework for machine learning [C]//Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security. ACM: 2018: 35–52. DOI: 10.1145/3243734.3243760
DOI |
27 | CHEN V, PASTRO V, RAYKOVA M. Secure computation for machine learning with SPDZ [EB/OL]. [2022-04-01]. |
28 |
BONAWITZ K, IVANOV V, KREUTER B, et al. Practical secure aggregation for privacy-preserving machine learning [C]//Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. ACM, 2017. DOI: 10.1145/3133956.3133982
DOI |
29 |
GENTRY C. Fully homomorphic encryption using ideal lattices [C]//Proceedings of the 41st Annual ACM Symposium on Theory of Computing. ACM, 2009: 169–178. DOI: 10.1145/1536414.1536440
DOI |
30 | GENTRY C. A fully homomorphic encryption scheme [M]. Stanford, USA: Stanford University, 2009 |
31 |
ACAR A, AKSU H, ULUAGAC A S, et al. A survey on homomorphic encryption schemes [J]. ACM computing surveys, 2019, 51(4): 1–35. DOI: 10.1145/3214303
DOI |
32 | ZHANG C, LI S, XIA J, et al. BatchCrypt: efficient homomorphic encryption for cross-silo federated learning [C]//2020 USENIX Annual Technical Conference. USENIX, 2020: 493–506 |
33 |
FANG H K, QIAN Q. Privacy preserving machine learning with homomorphic encryption and federated learning [J]. Future Internet, 2021, 13(4): 94. DOI: 10.3390/fi13040094
DOI |
34 | WANG J, LIU Q, LIANG H, et al. Tackling the objective inconsistency problem in heterogeneous federated optimization [J]. Advances in neural information processing systems, 2020, 33: 7611–7623 |
35 | LI T, SAHU A K, ZAHEER M, et al. Federated optimization in heterogeneous networks [EB/OL]. [2022-04-01]. |
36 | KARIMIREDDY S P, KALE S, MOHRI M, et al. SCAFFOLD: stochastic controlled averaging for on-device federated learning [EB/OL]. [2022-04-01]. |
37 | HE C Y, ANNAVARAM M, AVESTIMEHR S. Towards non-IID and invisible data with FedNAS: federated deep learning via neural architecture search [EB/OL]. [2022-04-01]. |
38 | BLANCHARD P, MHAMDI E M EL, GUERRAOUI R, et al. Machine learning with adversaries: Byzantine tolerant gradient descent [C]//The 31st International Conference on Neural Information Processing Systems. ACM, 2017 |
39 | REISIZADEH A, FARNIA F, PEDARSANI R, et al. Robust federated learning: the case of affine distribution shifts [J]. Advances in neural information processing systems, 2020, 33: 21554–21565 |
40 | PARK J, HAN D J, CHOI M, et al. Sageflow: robust federated learning against both stragglers and adversaries [J]. Advances in neural information processing systems, 2021, 34: 840–851 |
41 |
PILLUTLA K, KAKADE S M, HARCHAOUI Z. Robust aggregation for federated learning [J]. IEEE transactions on signal processing, 2022, 70: 1142–1154. DOI: 10.1109/TSP.2022.3153135
DOI |
42 | LI S, CHENG Y, WANG W, et al. Learning to detect malicious clients for robust federated learning [EB/OL]. [2022-04-01]. |
43 |
SATTLER F, WIEDEMANN S, MULLER K R, et al. Robust and communication-efficient federated learning from non-IID data [J]. IEEE transactions on neural networks and learning systems, 2020, 31(9): 3400–3413. DOI: 10.1109/TNNLS.2019.2944481
DOI |
44 |
NISHIO T, YONETANI R. Client selection for federated learning with heterogeneous resources in mobile edge [C]//2019 IEEE International Conference on Communications. IEEE, 2019 : 1–7. DOI: 10.1109/ICC.2019.8761315
DOI |
45 | GEYER R C, KLEIN T, NABI M. Differentially private federated learning: A client level perspective [EB/OL]. [2022-04-01]. |
46 |
KANG J W, XIONG Z H, NIYATO D, et al. Incentive design for efficient federated learning in mobile networks: a contract theory approach [C]//2019 IEEE VTS Asia Pacific Wireless Communications Symposium. IEEE, 2019: 1–5. DOI: 10.1109/VTS-APWCS.2019.8851649
DOI |
47 | CHEN W, HORVATH S, RICHTARIK P. Optimal client sampling for federated learning [EB/OL]. [2022-04-01]. |
48 | KONEČNÝ J, MCMAHAN H B, YU F X, et al. Federated learning: strategies for improving communication efficiency [EB/OL]. [2022-04-01]. |
49 |
TRAN N H, BAO W, ZOMAYA A, et al. Federated learning over wireless networks: optimization model design and analysis [C]//IEEE Conference on Computer Communications. IEEE, 2019: 1387–1395. DOI: 10.1109/INFOCOM.2019.8737464
DOI |
50 | GUHA N, TALWALKAR A, SMITH V. One-shot federated learning [EB/OL]. [2022-04-01]. |
51 | CHOI B, SOHN J Y, HAN D J, et al. Communication-computation efficient secure aggregation for federated learning [EB/OL]. [2022-04-01]. |
52 | DU W, ZENG X, YAN M, et al. Efficient federated learning via variational dropout [EB/OL]. [2022-04-01]. |
53 |
REN J J, WANG H C, HOU T T, et al. Federated learning-based computation offloading optimization in edge computing-supported Internet of Things [J]. IEEE access, 7: 69194–69201. DOI: 10.1109/ACCESS.2019.2919736
DOI |
54 |
YU H, LIU Z L, LIU Y, et al. A fairness-aware incentive scheme for federated learning [C]//Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. ACM, 2020: 393–399. DOI: 10.1145/3375627.3375840
DOI |
55 | LI T, SANJABI M, SMITH V. Fair resource allocation in federated learning [EB/OL]. [2022-04-01]. |
56 |
LYU L, XU X, WANG Q, et al. Collaborative fairness in federated learning [M]//Federated Learning. Cham: Springer, 2020: 189–204. DOI: 10.1007/978-3-030-63076-8_14
DOI |
57 | INGERMAN A, OSTROWSKI K. TensorFlow federated [EB/OL]. [2022-04-01]. |
58 | LIU Y, FAN T, CHEN T, et al. FATE: an industrial grade platform for collaborative learning with data protection [J]. Journal of machine learning research, 2021, 22(226): 1–6 |
59 | ZILLER A, TRASK A, LOPARDO A, et al. Pysyft: a library for easy federated learning [M]//Federated learning systems. Cham: Springer, 2021: 111–139 |
60 | HE C Y, LI S Z, SO J, et al. FedML: a research library and benchmark for federated machine learning [EB/OL]. [2022-04-01]. |
61 | MA Y, YU D, WU T, et al. PaddlePaddle: an open-source deep learning platform from industrial practice [J]. Frontiers of data and computing, 2019, 1(1): 105–115 |
[1] | YAN Yuna, LIU Ying, NI Tao, LIN Wensheng, LI Lixin. Content Popularity Prediction via Federated Learning in Cache-Enabled Wireless Networks [J]. ZTE Communications, 2023, 21(2): 18-24. |
[2] | ZHAO Moke, HUANG Yansong, LI Xuan. Federated Learning for 6G: A Survey From Perspective of Integrated Sensing, Communication and Computation [J]. ZTE Communications, 2023, 21(2): 25-33. |
[3] | ZHANG Weiting, LIANG Haotian, XU Yuhua, ZHANG Chuan. Reliable and Privacy-Preserving Federated Learning with Anomalous Users [J]. ZTE Communications, 2023, 21(1): 15-24. |
[4] | WANG Yiji, WEN Dingzhu, MAO Yijie, SHI Yuanming. RIS-Assisted Federated Learning in Multi-Cell Wireless Networks [J]. ZTE Communications, 2023, 21(1): 25-37. |
[5] | TONG Ze, DENG Bowen, ZHENG Lele, ZHANG Tao. Utility-Improved Key-Value Data Collection with Local Differential Privacy for Mobile Devices [J]. ZTE Communications, 2022, 20(4): 15-21. |
[6] | WANG Pengfei, SONG Wei, SUN Geng, WEI Zongzheng, ZHANG Qiang. Air-Ground Integrated Low-Energy Federated Learning for Secure 6G Communications [J]. ZTE Communications, 2022, 20(4): 32-40. |
[7] | NAN Yucen, FANG Minghao, ZOU Xiaojing, DOU Yutao, Albert Y. ZOMAYA. A Collaborative Medical Diagnosis System Without Sharing Patient Data [J]. ZTE Communications, 2022, 20(3): 3-16. |
[8] | HAN Xuming, GAO Minghan, WANG Limin, HE Zaobo, WANG Yanze. A Survey of Federated Learning on Non-IID Data [J]. ZTE Communications, 2022, 20(3): 17-26. |
[9] | LIU Qinbo, JIN Zhihao, WANG Jiabo, LIU Yang, LUO Wenjian. MSRA-Fed: A Communication-Efficient Federated Learning Method Based on Model Split and Representation Aggregate [J]. ZTE Communications, 2022, 20(3): 35-42. |
[10] | SHI Wenqi, SUN Yuxuan, HUANG Xiufeng, ZHOU Sheng, NIU Zhisheng. Scheduling Policies for Federated Learning in Wireless Networks: An Overview [J]. ZTE Communications, 2020, 18(2): 11-19. |
[11] | YANG Howard H., ZHAO Zhongyuan, QUEK Tony Q. S.. Enabling Intelligence at Network Edge:An Overview of Federated Learning [J]. ZTE Communications, 2020, 18(2): 2-10. |
[12] | Xinhua Dong, Ruixuan Li, Wanwan Zhou, Dongjie Liao, and Shuoyi Zhao. Data Security and Privacy in Cloud Storage [J]. ZTE Communications, 2013, 11(2): 18-23. |
Viewed | ||||||
Full text |
|
|||||
Abstract |
|
|||||