Scheduling Policies for Federated Learning in Wireless Networks: An Overview

doi:10.12142/ZTECOM.202002003

ZTE Communications ›› 2020, Vol. 18 ›› Issue (2): 11-19.doi: 10.12142/ZTECOM.202002003

• Special Topic • Previous Articles Next Articles

Scheduling Policies for Federated Learning in Wireless Networks: An Overview

SHI Wenqi, SUN Yuxuan, HUANG Xiufeng, ZHOU Sheng(), NIU Zhisheng

Department of Electronic Engineering, Tsinghua University, Beijing 100084, China

Received:2020-02-10 Online:2020-06-25 Published:2020-08-07
About author:SHI Wenqi received his B.S. degree in electronic engineering from Tsinghua University, China in 2017. He is pursuing his Ph.D. degree in electronic engineering with Tsinghua University. His research interests include edge computing, machine learning and machine learning applications in wireless communications.|SUN Yuxuan received her B.S. degree in telecommunications engineering from Tianjin University, China, in 2015. She is currently working toward the Ph.D. degree in electronic engineering with Tsinghua University. Her research interests include mobile edge computing, vehicular cloud computing and distributed machine learning.|HUANG Xiufeng received his B.S. degree in electronic engineering from Tsinghua University, China, in 2018. He is currently a Ph.D. student in electronic engineering with Tsinghua University. His research interests include machine learning, edge computing and performance optimization for machine learning applications in wireless networks.|ZHOU Sheng (sheng.zhou@tsinghua.edu.cn) received his B.S. and Ph.D. degrees in electronic engineering from Tsinghua University, China, in 2005 and 2011, respectively. He is currently an associate professor of Electronic Engineering Department, Tsinghua University. His research interests include cross-layer design for multiple antenna systems, vehicular networks, mobile edge computing and green wireless communications.|NIU Zhisheng graduated from Beijing Jiaotong University, China, in 1985, and got his M.E. and D.E. degrees from Toyohashi University of Technology, Japan, in 1989 and 1992, respectively. During 1992–1994, he worked for Fujitsu Laboratories Ltd., Japan, and in 1994, he joined in Tsinghua University, China, where he is now a professor at the Department of Electronic Engineering. His major research interests include queueing theory, traffic engineering, mobile Internet, radio resource management of wireless networks, and green communication and networks.
Supported by:
the National Key R&D Program of China(2018YFB1800800);the Nature Science Foundation of China(61871254)

Abstract

Abstract:

Due to the increasing need for massive data analysis and machine learning model training at the network edge, as well as the rising concerns about data privacy, a new distributed training framework called federated learning (FL) has emerged and attracted much attention from both academia and industry. In FL, participating devices iteratively update the local models based on their own data and contribute to the global training by uploading model updates until the training converges. Therefore, the computation capabilities of mobile devices can be utilized and the data privacy can be preserved. However, deploying FL in resource-constrained wireless networks encounters several challenges, including the limited energy of mobile devices, weak onboard computing capability, and scarce wireless bandwidth. To address these challenges, recent solutions have been proposed to maximize the convergence rate or minimize the energy consumption under heterogeneous constraints. In this overview, we first introduce the backgrounds and fundamentals of FL. Then, the key challenges in deploying FL in wireless networks are discussed, and several existing solutions are reviewed. Finally, we highlight the open issues and future research directions in FL scheduling.

Key words: federated learning, wireless network, edge computing, scheduling

SHI Wenqi, SUN Yuxuan, HUANG Xiufeng, ZHOU Sheng, NIU Zhisheng. Scheduling Policies for Federated Learning in Wireless Networks: An Overview[J]. ZTE Communications, 2020, 18(2): 11-19.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

Figures/Tables 6

Figure 1

Table 1

Figure 2

Figure 3

Table 2

Figure 4

References 38

1	CHIANG M, ZHANG T. Fog and IoT: an overview of research opportunities [J]. IEEE internet of things journal, 2016, 3(6): 854–864. DOI: 10.1109/JIOT.2016.2584538 doi: 10.1109/JIOT.2016.2584538
2	ZHU G, LIU D, DU Y, et al. Towards an intelligent edge: wireless communication meets machine learning [EB/OL]. (2018⁃09⁃02)[2020⁃01⁃31].
3	PARK J, SAMARAKOON S, BENNIS M, et al. Wireless network intelligence at the edge [J]. Proceedings of the IEEE, 2019, 107(11): 2204–2239. DOI: 10.1109/JPROC.2019.2941458 doi: 10.1109/JPROC.2019.2941458
4	LIM W Y, LUONG N C, HOANG D T, et al. Federated learning in mobile edge networks: a comprehensive survey [EB/OL]. (2019⁃09⁃26)[2020⁃01⁃31].
5	BRISIMI T S, CHEN R, MELA T, et al. Federated learning of predictive models from federated electronic health records [J]. International journal of medical informatics, 2018, 112: 59–67. DOI: 10.1016/j.ijmedinf.2018.01.007 doi: 10.1016/j.ijmedinf.2018.01.007
6	MELIS L, SONG C, CRISTOFARO E DE, et al. Exploiting unintended feature leakage in collaborative learning [EB/OL]. (2018⁃05⁃10)[2019⁃11⁃01].
7	ABADI M, CHU A, GOODFELLOW I, et al. Deep learning with differential privacy [C]//Proceedings of 2016 Acm Sigsac Conference on Computer And Communications Security. New York, USA: ACM, 2016: 308–318. DOI: 10.1145/2976749.2978318 doi: 10.1145/2976749.2978318
8	GEYER R C, KLEIN T, NABI M. Differentially private federated learning: a client level perspective [EB/OL]. (2018⁃03⁃01)[2020⁃01⁃31].
9	SHOKRI R, SHMATIKOV V. Privacy⁃Preserving Deep Learning [C]//Proceedings of 22nd ACM Sigsac Conference on Computer and Communications Security. New York, USA: ACM, 2015: 1310–1321. DOI: 10.1145/2810103.2813687 doi: 10.1145/2810103.2813687
10	LIU Y, MA Z, MA S, et al. Boosting privately: privacy⁃preserving federated extreme boosting for mobile crowdsensing [EB/OL]. (2019⁃07⁃24) [2020⁃01⁃31].
11	AONO Y, HAYASHI T, WANG L, et al. Privacy⁃preserving deep learning via additively homomorphic encryption [J]. IEEE transactions on information forensics and security, 2017, 13(5): 1333–1345. DOI: 10.1109/TIFS.2017.2787987 doi: 10.1109/TIFS.2017.2787987
12	HAO M, LI H W, XU G W, et al. Towards efficient and privacy⁃preserving federated deep learning [C]//IEEE International Conference on Communications (ICC). Shanghai, China, 2019: 1–6. DOI: 10.1109/icc.2019.8761267 doi: 10.1109/icc.2019.8761267
13	KONEČNÝ J, MCMAHAN B, RAMAGE D. Federated optimization: distributed optimization beyond the datacenter [EB/OL]. (2015⁃11⁃11)[2020⁃01⁃31].
14	AJI A F, HEAFIELD K. Sparse communication for distributed gradient descent [C]//Proceedings of 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: Association for Computational Linguistics, 2017. DOI: 10.18653/v1/d17⁃1045 doi: 10.18653/v1/d17?1045
15	CHEN T Y, GIANNAKIS G B, SUN T, et al. LAG: Lazily aggregated gradient for communication⁃efficient distributed learning [C]//Adavances in Neural Information Processing Systems 31 (NeurIPS 2018). Montreal, Canada, 2018: 5055–5065
16	SHI W, ZHOU S, NIU Z. Device scheduling with fast convergence for wireless federated learning [EB/OL]. (2019⁃11⁃03)[2020⁃01⁃31].
17	ZENG Q, DU Y, LEUNG K K, et al. Energy⁃efficient radio resource allocation for federated edge learning [EB/OL]. (2019⁃07⁃13)[2020⁃01⁃31].
18	LI T, SAHU A K, TALWALKAR A, et al. Federated learning: challenges, methods, and future directions [EB/OL]. (2019⁃8⁃21)[2020⁃01⁃31].
19	WANG X, HAN Y, LEUNG V C M, et al. Convergence of edge computing and deep learning: a comprehensive survey [J]. IEEE communications surveys & tutorials, 2020. DOI: 10.1109/COMST.2020.2970550 doi: 10.1109/COMST.2020.2970550
20	SHI Y, YANG K, JIANG T, et al. Communication⁃efficient edge AI: algorithms and systems [EB/OL]. (2020⁃02⁃22).
21	LIM W Y B, LUONG N C, HOANG D T, et al. Federated learning in mobile edge networks: a comprehensive survey [EB/OL]. (2020⁃02⁃28).
22	GUNDUZ D, DE KERRET P, SIDIROPOULOS N D, et al. Machine learning in the air [J]. IEEE journal on selected areas in communications, 2019, 37(10): 2184–2199. DOI: 10.1109/JSAC.2019.2933969 doi: 10.1109/JSAC.2019.2933969
23	ZHU G X, WANG Y, HUANG K B. Broadband analog aggregation for low⁃latency federated edge learning [J]. IEEE transactions on wireless communications, 2020, 19(1): 491–506. DOI: 10.1109/TWC.2019.2946245 doi: 10.1109/TWC.2019.2946245
24	SUN Y, ZHOU S, GUNDUZ D. Energy⁃aware analog aggregation for federated learning with redundant data [EB/OL]. (2019⁃11⁃01)[2020⁃01⁃31].
25	YANG K, JIANG T, SHI Y, et al. Federated learning via over⁃the⁃air computation [EB/OL]. (2019⁃02⁃17)[2020⁃01⁃31].
26	AMIRI M M, GUNDUZ D. Machine learning at the wireless edge: distributed stochastic gradient descent over⁃the⁃air [C]//IEEE International Symposium on Information Theory (ISIT). Paris, France, 2019: 1432–1436. DOI:10.1109/isit.2019.8849334 doi: 10.1109/isit.2019.8849334
27	AMIRI M M, GUNDUZ D. Federated learning over wireless fading channels [EB/OL]. (2019⁃07⁃23)[2020⁃01⁃31].
28	ZHAO Y, LI M, LAI L, et al. Federated learning with non⁃iid data [EB/OL]. (2018⁃06⁃02)[2020⁃01⁃31].
29	WANG S Q, TUOR T, SALONIDIS T, et al. Adaptive federated learning in resource constrained edge computing systems [J]. IEEE journal on selected areas in communications, 2019, 37(6): 1205–1221. DOI: 10.1109/jsac.2019.2904348 doi: 10.1109/jsac.2019.2904348
30	LIU L, ZHANG J, SONG S H, et al. Edge⁃assisted hierarchical federated learning with non⁃iid data [EB/OL]. (2019⁃10⁃31)[2020⁃01⁃31].
31	TRAN N H, BAO W, ZOMAYA A, et al. Federated learning over wireless networks: optimization model design and analysis [C]//IEEE Conference on Computer Communications. Paris, France, 2019: 1387–1395. DOI: 10.1109/INFOCOM.2019.8737464 doi: 10.1109/INFOCOM.2019.8737464
32	YANG Z, CHEN M, SAAD W, et al. Energy efficient federated learning over wireless communication networks [EB/OL]. (2019⁃11⁃6)[2020⁃01⁃31].
33	BONAWITZ K, EICHNER H, GRIESKAMP W, et al. Towards federated learning at scale: system design [EB/OL]. (2019⁃3⁃22)[2020⁃01⁃31].
34	STICH S U. Local SGD converges fast and communicates little [EB/OL]. (2019⁃05⁃03)[2020⁃01⁃31].
35	YANG H H, LIU Z Z, QUEK T Q S, et al. Scheduling policies for federated learning in wireless networks [J]. IEEE transactions on communications, 2020, 68(1): 317–333. DOI: 10.1109/tcomm.2019.2944169 doi: 10.1109/tcomm.2019.2944169
36	CHEN M, YANG Z, SAAD W, et al. A joint learning and communications framework for federated learning over wireless networks [EB/OL]. (2019⁃9⁃17)[2020⁃01⁃31].
37	NISHIO T, YONETANI R. Client selection for federated learning with heterogeneous resources in mobile edge [C]//ICC 2019⁃2019 IEEE International Conference On Communications (ICC). Shanghai, China, 2019: 1⁃7. DOI: 10.1109/ICC.2019.8761315 doi: 10.1109/ICC.2019.8761315
38	NISHIO T, YONETANI R. Client selection for federated learning with heterogeneous resources in mobile edge [C]//IEEE International Conference on Communications (ICC). Shanghai, China, 2019: 1–7. DOI: 10.1109/icc.2019.8761315 doi: 10.1109/icc.2019.8761315

Technology	Highlights	Related Works
Power alignment	· Fundamental tradeoffs under Rayleigh fading channel	Ref. [23]
	· Online energy-aware dynamic device scheduling policy	Ref. [24]
	· Device scheduling for multi-antenna analog aggregation	Ref. [25]
Sparsification and error accumulation	· Gradient sparsification and error accumulation · Device scheduling policy under average power constraint	Refs. [26–27]
Data redundancy	· Introducing data redundancy to deal with non-independent and identically distributed (non-i.i.d.) data	Ref. [24]

Technology	Highlights	Related Works
Aggregation frequency adaption	· Global aggregation frequency adaption under given resource constraints.	Ref. [29]
Aggregation frequency adaption	· Extending Ref. [29] into a client-edge-cloud hierarchical FL system	Ref. [30]
Local accuracy tuning	· Tuning local model accuracy to balance the tradeoff between local update and global aggregation · Energy- and convergence-aware resource allocation	Refs. [31–32]
Device scheduling	· Energy- and convergence-aware joint scheduling and resource allocation	Ref. [17]
	·Consider unreliable wireless transmissions	Refs. [35–36]
	· Maximize the convergence rate with respect to time	Refs. [16] and [37]

[1]	ZHAO Kongyang, GAO Bin, ZHOU Zhi. Cost-Effective Task Scheduling for Collaborative Cross-Edge Analytics [J]. ZTE Communications, 2021, 19(2): 11-19.
[2]	YANG Howard H., ZHAO Zhongyuan, QUEK Tony Q. S.. Enabling Intelligence at Network Edge:An Overview of Federated Learning [J]. ZTE Communications, 2020, 18(2): 2-10.
[3]	WU Hequan. Ten Reflections on 5G [J]. ZTE Communications, 2020, 18(1): 1-4.
[4]	Mohammed SEID, Stephen ANOKYE, SUN Guolin. Machine Learning Based Unmanned Aerial Vehicle Enabled Fog-Radio Aerial Vehicle Enabled Fog-Radio Access Network and Edge Computing [J]. ZTE Communications, 2019, 17(4): 33-45.
[5]	CAO Jie, XU Lanyu, Raef Abdallah, SHI Weisong. An OS for Internet of Everything: Early Experience from A Smart Home Prototype [J]. ZTE Communications, 2017, 15(4): 12-22.
[6]	ZHOU Yuezhi, ZHANG Di, ZHANG Yaoxue. A Transparent and User-Centric Approach to Unify Resource Management and Code Scheduling of Local, Edge, and Cloud [J]. ZTE Communications, 2017, 15(4): 3-11.
[7]	TU Yaofeng, DONG Zhenjiang, YANG Hongzhang. Key Technologies and Application of Edge Computing [J]. ZTE Communications, 2017, 15(2): 26-34.
[8]	Panagiotis Oikonomou, Maria G. Koziri, Nikos Tziritas, Thanasis Loukopoulos, XU Cheng-Zhong. Scheduling Heuristics for Live Video Transcoding on Cloud Edges [J]. ZTE Communications, 2017, 15(2): 35-41.
[9]	Borui Ren, Gang Liu, Bin Hou. Utility-Based Joint Scheduling Approach Supporting Multiple Services for CoMP-SU-MIMO in LTE-A System [J]. ZTE Communications, 2015, 13(1): 60-66.
[10]	Lianming Zhang, Jia Liu, and Kun Yang. VirtualizedWireless SDNs: Modelling Delay Through the Use of Stochastic Network Calculus [J]. ZTE Communications, 2014, 12(2): 50-56.
[11]	Tung Nguyen and Weisong Shi. MapReduce in the Cloud: Data-Location-Aware VM Scheduling [J]. ZTE Communications, 2013, 11(4): 18-26.
[12]	Shihao Yan and Robert Malaney. Location Verification Systems in Emerging Wireless Networks [J]. ZTE Communications, 2013, 11(3): 3-10.
[13]	Shih-Hao Hung, Tei-Wei Kuo, Chi-Sheng Shih, Jeng-Peng Shieh, Chen-Pang Lee, Che-Wei Chang, and Jie-Wen Wei. A Cloud-Based Virtualized Execution Environment for Mobile Applications [J]. ZTE Communications, 2011, 9(1): 15-21.
[14]	Liu Fuqiang, Shan Lianhai. Heterogeneous Vehicular Communication Architecture and Key Technologies [J]. ZTE Communications, 2010, 8(4): 39-44.

Scheduling Policies for Federated Learning in Wireless Networks: An Overview

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 6

References 38

Related Articles 14

Recommended Articles

Metrics

Comments