Enabling Intelligence at Network Edge:An Overview of Federated Learning

doi:10.12142/ZTECOM.202002002

ZTE Communications ›› 2020, Vol. 18 ›› Issue (2): 2-10.DOI: 10.12142/ZTECOM.202002002

收稿日期:2020-02-10 出版日期:2020-06-25 发布日期:2020-08-07

Enabling Intelligence at Network Edge:An Overview of Federated Learning

YANG Howard H.¹, ZHAO Zhongyuan²(), QUEK Tony Q. S.¹

^1.Singapore University of Technology and Design, Singapore 487372, Singapore
^2.Beijing University of Post and Telecommunication, Beijing 100876, China

Received:2020-02-10 Online:2020-06-25 Published:2020-08-07
About author:Howard H. YANG received the B.Sc. degree in communication engineering from Harbin Institute of Technology (HIT), China, in 2012, the M.Sc. degree in electronic engineering from Hong Kong University of Science and Technology (HKUST), China, in 2013, and the Ph.D. degree in electronic engineering from Singapore University of Technology and Design (SUTD), Singapore, in 2017. His background also features appointments at the University of Texas at Austin, USA and Princeton University, USA. His research interests cover various aspects of wireless communications, networking and signal processing, currently focusing on the modeling of modern wireless networks, high dimensional statistics, graph signal processing and machine learning. He received the IEEE WCSP 10-Year Anniversary Excellent Paper Award in 2019 and the IEEE WCSP Best Paper Award in 2014.|ZHAO Zhongyuan (zyzhao@bupt.edu.cn） received the B.S. and Ph.D. degrees from Beijing University of Posts and Telecommunications (BUPT), China, in 2009 and 2014, respectively. He is currently an associate professor with BUPT. His research interests include mobile cloud and fog computing and network edge intelligence. Dr. ZHAO serves as an editor of IEEE Communications Letters (since 2016). He was the recipient of the Best Paper Awards at the IEEE CIT 2014 and WASA 2015. He was also the recipient of Exemplary Reviewers-2017 of IEEE Transactions on Communications, and Exemplary Editor Award 2017 and 2018 of IEEE Communication Letters.|Tony Q. S. QUEK received the B.E. and M.E. degrees in electrical and electronics engineering from Tokyo Institute of Technology, Japan. At MIT, USA, he earned the Ph.D. in electrical engineering and computer science. Currently, he is the Cheng Tsang Man Chair Professor with Singapore University of Technology and Design (SUTD). He also serves as the acting head of Information System Technology and Design (ISTD) Pillar, sector lead for SUTD AI Program, and the deputy director of SUTD-ZJU IDEA. He is currently serving as an editor for the IEEE Transactions on Wireless Communications, the chair of IEEE VTS Technical Committee on Deep Learning for Wireless Communications as well as an elected member of the IEEE Signal Processing Society SPCOM Technical Committee. He received the 2008 Philip Yeo Prize for Outstanding Achievement in Research, the 2012 IEEE William R. Bennett Prize, the 2016 IEEE Signal Processing Society Young Author Best Paper Award, 2017 CTTC Early Achievement Award, 2017 IEEE ComSoc AP Outstanding Paper Award, and 2016-2019 Clarivate Analytics Highly Cited Researcher. He is a Distinguished Lecturer of the IEEE Communications Society and a Fellow of IEEE.

摘要/Abstract

Abstract:

The burgeoning advances in machine learning and wireless technologies are forging a new paradigm for future networks, which are expected to possess higher degrees of intelligence via the inference from vast dataset and being able to respond to local events in a timely manner. Due to the sheer volume of data generated by end-user devices, as well as the increasing concerns about sharing private information, a new branch of machine learning models, namely federated learning, has emerged from the intersection of artificial intelligence and edge computing. In contrast to conventional machine learning methods, federated learning brings the models directly to the device for training, where only the resultant parameters shall be sent to the edge servers. The local copies of the model on the devices bring along great advantages of eliminating network latency and preserving data privacy. Nevertheless, to make federated learning possible, one needs to tackle new challenges that require a fundamental departure from standard methods designed for distributed optimizations. In this paper, we aim to deliver a comprehensive introduction of federated learning. Specifically, we first survey the basis of federated learning, including its learning structure and the distinct features from conventional machine learning models. We then enumerate several critical issues associated with the deployment of federated learning in a wireless network, and show why and how technologies should be jointly integrated to facilitate the full implementation from different perspectives, ranging from algorithmic design, on-device training, to communication resource management. Finally, we conclude by shedding light on some potential applications and future trends.

Key words: federated learning, edge intelligence, learning algorithm, communication efficiency, privacy and security

. [J]. ZTE Communications, 2020, 18(2): 2-10.

YANG Howard H., ZHAO Zhongyuan, QUEK Tony Q. S.. Enabling Intelligence at Network Edge:An Overview of Federated Learning[J]. ZTE Communications, 2020, 18(2): 2-10.

图/表 4

Figure 1 Illustration of the network architecture, in which a mobile edge system is integrated with federated learning.

Figure 2 Test accuracy of federated learning under different scheduling policies.

Figure 3 Basic flow of model compression in the federated learning system.

Figure 4 Performance of federated learning with different malicious users.

参考文献 24

1	MAO Y Y, YOU C S, ZHANG J, et al. A survey on mobile edge computing: the communication perspective [J]. IEEE communications surveys & tutorials, 2017, 19(4): 2322–2358. DOI: 10.1109/comst.2017.2745201 DOI
2	LETAIEF K B, CHEN W, SHI Y M, et al. The roadmap to 6G: AI empowered wireless networks [J]. IEEE communications magazine, 2019, 57(8): 84–90
3	KONEČNÝ J, MCMAHAN H B, YU F, et al. Federated learning: strategies for improving communication efficiency [EB/OL]. (2016⁃10⁃18) [2019⁃09⁃17].
4	MCMAHAN H B, MOORE E, RAMAGE D, et al. Communication⁃efficient learning of deep networks from decentralized data [EB/OL]. (2016⁃02⁃17) [2019⁃09⁃17].
5	SMITH V, FORTE S, MA C X, et al. CoCoA: a general framework for communication⁃efficient distributed optimization [J]. Journal of machine learning research, 2018, 18(230): 1–49
6	PARK J, SAMARAKOON S, BENNIS M, et al. Wireless network intelligence at the edge [J]. Proceedings of the IEEE, 2019, 107(11): 2204–2239. DOI: 10.1109/jproc.2019.2941458 DOI
7	ZHAO Z Y, FENG C Y, YANG H H, et al. Federated⁃learning⁃enabled intelligent fog radio access networks: fundamental theory, key techniques, and future trends [J]. IEEE wireless communications, 2020, 27(2): 22–28. DOI: 10.1109/mwc.001.1900370 DOI
8	ZHOU Z, CHEN X, LI E, et al. Edge intelligence: paving the last mile of artificial intelligence with edge computing [J]. Proceedings of the IEEE, 2019, 107(8): 1738–1762. DOI: 10.1109/jproc.2019.2918951 DOI
9	ZHU G X, LIU D Z, DU Y Q, et al. Toward an intelligent edge: wireless communication meets machine learning [J]. IEEE communications magazine, 2020, 58(1): 19–25. DOI: 10.1109/mcom.001.1900103 DOI
10	KAIROUZ P, MCMAHAN H B, AVENTET B, et al. Advances and open problems in federated learning [EB/OL]. (2019⁃12⁃10) [2019⁃09⁃17].
11	WANG S Q, TUOR T, SALONIDIS T, et al. Adaptive federated learning in resource constrained edge computing systems [J]. IEEE journal on selected areas in communications, 2019, 37(6): 1205–1221
12	YANG H H, LIU Z, QUEK T Q S, et al. Scheduling policies for federated learning in wireless networks [J]. IEEE transactions on communications, 2020, 68(1): 317–333
13	DAI W, ZHOU Y, DONG N Qet al. Toward understanding the impact of staleness in distributed machine learning [C]//International Conference for Learning Representations (ICLR). New Orleans, Louisiana, 2019: 1–6
14	MA C, LI J, DING M, et al. On safeguarding privacy and security in the framework of federated learning [J]. IEEE network, 2020: 1–7. DOI: 10.1109/mnet.001.1900506 DOI
15	TRAN N H, BAO W, ZOMAYA A, et al. Federated learning over wireless networks: optimization model design and analysis [C]//IEEE Conference on Computer Communications (INFOCOM). Paris, France, 2019. DOI: 10.1109/infocom.2019.8737464 DOI
16	CHEN M Z, YANG Z H, SAAD W, et al. A joint learning and communications framework for federated learning over wireless networks [EB/OL]. [2019⁃09⁃17].
17	YANG H H, ARAFA A, QUEK T Q S, et al. Age⁃based scheduling policy for federated learning in mobile edge networks [C]//IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Barcelona, Spain, 2020. DOI: 10.1109/icassp40776.2020.9053740 DOI
18	DU Y Q, YANG S, HUANG K B. High⁃dimensional stochastic gradient quantization for communication⁃efficient edge learning [J]. IEEE transactions on signal processing, 2020, 68: 2128–2142.
19	ZHU G X, DU Y Q, GÜNDÜZ D, et al. One⁃bit over⁃the⁃air aggregation for communication⁃efficient federated edge learning: design and convergence analysis [EB/OL]. [2020⁃01⁃16].
20	ZHU G X, WANG Y, HUANG K B. Broadband analog aggregation for low⁃latency federated edge learning [J]. IEEE transactions on wireless communications, 2020, 19(1): 491–506. DOI: 10.1109/twc.2019.2946245 DOI
21	YANG K, JIANG T, SHI Y M, et al. Federated learning via over⁃the⁃air computation [J]. IEEE transactions on wireless communications, 2020, 19(3): 2022–2035. DOI: 10.1109/twc.2019.2961673 DOI
22	AMIRI M M, GUNDUZ D. Machine learning at the wireless edge: distributed stochastic gradient descent over⁃the⁃air [J]. IEEE transactions on signal processing, 2020, 68: 2155–2169
23	PHONG L T, AONO Y, HAYASHI T, et al. Privacy⁃preserving deep learning via additively homomorphic encryption [J]. IEEE transactions on information forensics and security, 2018, 13(5): 1333–1345. DOI: 10.1109/tifs.2017.2787987 DOI
24	DWORK C, MCSHERRY F, NISSIM K, et al. Calibrating noise to sensitivity in private data analysis [M]//Theory of cryptography. Berlin, Heidelberg, Germany: Springer Berlin Heidelberg, 2006: 265–284. DOI: 10.1007/11681878_14 DOI

Enabling Intelligence at Network Edge:An Overview of Federated Learning

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 4

参考文献 24

相关文章 0

编辑推荐

Metrics