A Survey on Task Scheduling of CPU-GPU Heterogeneous Cluster

doi:10.12142/ZTECOM.202403010

ZTE Communications ›› 2024, Vol. 22 ›› Issue (3): 83-90.DOI: 10.12142/ZTECOM.202403010

• Review • Previous Articles Next Articles

A Survey on Task Scheduling of CPU-GPU Heterogeneous Cluster

ZHOU Yiheng¹, ZENG Wei²(), ZHENG Qingfang³^,⁴, LIU Zhilong³^,⁴, CHEN Jianping²

^1.University of Shanghai for Science and Technology, Shanghai 200093, China
^2.National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University, Beijing 100871, China
^3.State Key Laboratory of Mobile Network and Mobile Multimedia Technology, Shenzhen 518055, China
^4.ZTE Corporation, Shenzhen 518057, China

Received:2024-04-07 Online:2024-09-25 Published:2024-09-29
About author:ZHOU Yiheng has received his bachelor’s degree in robotic engineering from University of Shanghai for Science and Technology, China in 2024. He has been working as a research assistant at Peking University, China since 2023. His research interests include computer vision (detection and pose estimation), robotic arm control, and artificial intelligence (AI computing platform).
ZENG Wei (weizeng@pku.edu.cn) is currently a research professor at the School of Computer Science, Peking University, China. He was a senior researcher at NEC Laboratories, China from 2005 to 2015. He worked as a visiting scholar at Stanford University, USA in 2012. He received his PhD degree in computer science and engineering from Harbin Institute of Technology, China in 2005. His research interests include computer vision (object detection and segmentation), artificial intelligence (AI computing platform), and media analysis (video retrieval). He is the author or coauthor of over 40 refereed journals and conference papers. He was the reviewer of ICCV2023, CVPR2023, ACM MM2022, ICPR2020, BigMM2020, and so forth.
ZHENG Qingfang received his BS degree from Shanghai Jiaotong University, China in 2002 and PhD degree in computer science from the Institute of Computing Technology, Chinese Academy of Sciences in 2008. He is now the chief scientist of cloud video product and deputy director of the Video Technology Committee of ZTE Corporation. His current research interests include video communication, computer vision and artificial intelligence.
LIU Zhilong is currently working with the Video Systems Department, ZTE Corporation as a senior system architecture engineer. He graduated from University of Electronic Science and Technology of China with a master’s degree in 2015. His research interests include media transmission, real-time audio and video networks (RTN), and video computing networks.
CHEN Jianping is currently a senior research engineer at the School of Computer Science, Peking University, China. His research interests include media analysis, AI platform, and video retrieval.
Supported by:
ZTE?University?Institute Fund Project(IA20230629009)

Abstract

Abstract:

This paper reviews task scheduling frameworks, methods, and evaluation metrics of central processing unit-graphics processing unit (CPU-GPU) heterogeneous clusters. Task scheduling of CPU-GPU heterogeneous clusters can be carried out on the system level, nodelevel, and device level. Most task-scheduling technologies are heuristic based on the experts’ experience, while some technologies are based on statistic methods using machine learning, deep learning, or reinforcement learning. Many metrics have been adopted to evaluate and compare different task scheduling technologies that try to optimize different goals of task scheduling. Although statistic task scheduling has reached fewer research achievements than heuristic task scheduling, the statistic task scheduling still has significant research potential.

Key words: CPU-GPU heterogeneous cluster, task scheduling, heuristic task scheduling, statistic task scheduling, parallelization

ZHOU Yiheng, ZENG Wei, ZHENG Qingfang, LIU Zhilong, CHEN Jianping. A Survey on Task Scheduling of CPU-GPU Heterogeneous Cluster[J]. ZTE Communications, 2024, 22(3): 83-90.

Figures/Tables 4

References 47

1	BOHN R B, MESSINA J, LIU F, et al. NIST cloud computing reference architecture [C]//IEEE World Congress on Services. IEEE, 2011: 594–596. DOI: 10.1109/SERVICES.2011.105
2	CAITHNESS N, DRESCHER M, WALLOM D. Can functional characteristics usefully define the cloud computing landscape and is the current reference model correct? [J]. Journal of cloud computing, 2017, 6(1): 10. DOI: 10.1186/s13677-017-0084-1
3	ARUNARANI A, MANJULA D, SUGUMARAN V. Task scheduling techniques in cloud computing: a literature survey [J]. Future generation computer systems, 2019, 91: 407–415. DOI: 10.1016/j.future.2018.09.014
4	SINGH P, DUTTA M, AGGARWAL N. A review of task scheduling based on meta-heuristics approach in cloud computing [J]. Knowledge and information systems, 2017, 52(1): 1–51. DOI: 10.1007/s10115-017-1044-2
5	PRADHAN R, SATAPATHY S C. Advances in intelligent systems and computing: task scheduling in heterogeneous cloud environment—a Survey [M]. Singapore: Springer, 2020
6	YANG G, ZHAO X, HUANG J. Survey on task scheduling algorithms for cloud computing [J]. Computer systems and applications, 2020, 29(3): 11–19, DOI: 10.15888/j.cnki.csa.007261
7	WANG H, WANG H H. Survey on task scheduling in cloud computing environment [C]//The 7th International Conference on Intelligent Informatics and Biomedical Science. IEEE, 2022: 286–291. DOI: 10.1109/ICIIBMS55689.2022.9971622
8	HOSSEINZADEH M, GHAFOUR M Y, HAMA H K, et al. Multi-objective task and workflow scheduling approaches in cloud computing: a comprehensive review [J]. Journal of grid computing, 2020, 18(3): 327–356. DOI: 10.1007/s10723-020-09533-z
9	PRITY F S, GAZI M H, ASLAM UDDIN K M. A review of task scheduling in cloud computing based on nature-inspired optimization algorithm [J]. Cluster computing, 2023, 26(5): 3037–3067. DOI: 10.1007/s10586-023-04090-y
10	JAWADE P B, KUMAR D SAI, RAMACHANDRAM S. A compact analytical survey on task scheduling in cloud computing environment [J]. International journal of engineering trends and technology, 2021, 69(2): 178–187. DOI: 10.14445/22315381/ijett-v69i2p225
11	SHIRAHATA K, SATO H, MATSUOKA S. Hybrid map task scheduling for GPU-based heterogeneous clusters [C]//The 2nd International Conference on Cloud Computing Technology and Science. IEEE, 2010: 733–740. DOI: 10.1109/CloudCom.2010.55
12	HUO H P, SHENG C C, HU X M, et al. An energy efficient task scheduling scheme for heterogeneous GPU-enhanced clusters [C]//International Conference on Systems and Informatics. IEEE, 2012: 623–627. DOI: 10.1109/ICSAI.2012.6223074
13	HU B, YANG X C, ZHAO M G. Energy-minimized scheduling of intermittent real-time tasks in a CPU-GPU cloud computing platform [J]. IEEE transactions on parallel and distributed systems, 2023, 34(8): 2391–2402. DOI: 10.1109/TPDS.2023.3288702
14	CAO Y P, WANG H F. A task scheduling scheme for preventing temperature hotspot on GPU heterogeneous cluster [C]//International Conference on Green Informatics. IEEE, 2017: 117–121. DOI: 10.1109/ICGI.2017.20
15	WANG H F, CAO Y P. Task scheduling of GPU cluster for large-scale data process with temperature constraint [C]//International Conference on Computer Engineering and Networks. CENet, 2020: 110-117.10.1007/978-3-030-14680-1_13
16	ZHANG K L, WU B F. Task scheduling for GPU heterogeneous cluster [C]//International Conference on Cluster Computing Workshops. IEEE, 2012: 161–169. DOI: 10.1109/ClusterW.2012.20
17	CHEN W B, YANG R R, YU J Q. Multi-granularity partition and scheduling for stream programs based on multi-CPU and multi-GPU heterogeneous architectures [J]. Computer engineering & science, 2017, 39(1): 15. DOI: 10.3969/j.issn.1007-130X.2017.01.002
18	CI Q Y, LI H R, YANG S W, et al. Adaptive and transparent task scheduling of GPU-powered clusters [J]. Concurrency and computation: Practice and experience, 2022, 34(9): e5793. DOI: 10.1002/cpe.5793
19	KEDAD-SIDHOUM S, MONNA F, MOUNIÉ G, et al. Scheduling independent tasks on multi-cores with GPU accelerators [C]//European Conference on Parallel Processing. Berlin, Heidelberg: Springer, 2014: 228-237.10.1007/978-3-642-54420-0_23
20	ZHU Z Y, TANG X C, ZHAO Q. A unified schedule policy of distributed machine learning framework for CPU-GPU cluster [J]. Journal of northwestern polytechnical university, 2021, 39(3): 529–538. DOI: 10.1051/jnwpu/20213930529
21	PINEL F, DORRONSORO B, BOUVRY P. Solving very large instances of the scheduling of independent tasks problem on the GPU [J]. Journal of parallel and distributed computing, 2013, 73(1): 101–110. DOI: 10.1016/j.jpdc.2012.02.018
22	SHAO J L, MA J M, LI Y, et al. GPU scheduling for short tasks in private cloud [C]//IEEE International Conference on Service-Oriented System Engineering. IEEE, 2019: 215–2155. DOI: 10.1109/SOSE.2019.00037
23	ZHANG W, LIAO X F, LI P, et al. Fine-grained scheduling in cloud gaming on heterogeneous CPU-GPU clusters [J]. IEEE network, 2018, 32(1): 172–178. DOI: 10.1109/MNET.2017.1700047
24	CHEN Z Y, LUO L, QUAN W, et al. Multiple CNN-based tasks scheduling across shared GPU platform in research and development scenarios [C]//The 20th International Conference on High Performance Computing and Communications. IEEE, 2018: 578–585. DOI: 10.1109/HPCC/SmartCity/DSS.2018.00107
25	CHEN Y W, HAN J C, ZHOU H, et al. GAS: GPU allocation strategy for deep learning training tasks [C]//IEEE Smartworld, Ubiquitous Intelligence & Computing, Scalable Computing & Communications, Digital Twin, Privacy Computing, Metaverse, Autonomous & Trusted Vehicles. IEEE, 2022: 880–887. DOI: 10.1109/SmartWorld-UIC-ATC-ScalCom-DigitalTwin-PriComp-Metaverse56740.2022.00133
26	HAN X C, JIANG W H, CAO P R, et al. Isolated scheduling for distributed training tasks in GPU clusters [EB/OL]. (2023-8-10)[2023-9-30].
27	CHEN Z Q, ZHAO X K, ZHI C, et al. DeepBoot: dynamic scheduling system for training and inference deep learning tasks in GPU cluster [J]. IEEE transactions on parallel and distributed systems, 2023, 34(9): 2553–2567. DOI: 10.1109/TPDS.2023.3293835
28	CHEN Z Y, QUAN W, WEN M, et al. Deep learning research and development platform: characterizing and scheduling with QoS guarantees on GPU clusters [J]. IEEE transactions on parallel and distributed systems, 2020, 31(1): 34–50. DOI: 10.1109/TPDS.2019.2931558
29	ZHANG K L, WU B F. Task scheduling greedy heuristics for GPU heterogeneous cluster involving the weights of the processor [C]//IEEE International Symposium on Parallel & Distributed Processing, Workshops and PhD Forum. IEEE, 2013: 1817–1827. DOI: 10.1109/IPDPSW.2013.38
30	ITURRIAGA S, NESMACHNOW S, LUNA F, et al. A parallel local search in CPU/GPU for scheduling independent tasks on large heterogeneous computing systems [J]. The journal of supercomputing, 2015, 71(2): 648–672. DOI: 10.1007/s11227-014-1315-6
31	GAO Y, GU W J, DING Y H, et al. Design and implementation of CPU and GPU cooperative scheduling algorithm with heterogeneous clusters [J]. Computer engineering and design, 2020, 41(2): 10. DOI: CNKI:SUN:SJSJ.0.2020-02-045
32	ZHANG H T, TANG B C, GENG X, et al. Learning driven parallelization for large-scale video workload in hybrid CPU-GPU cluster [C]//The 47th International Conference on Parallel Processing. ACM, 2018. DOI: 10.1145/3225058.3225070
33	ZHANG H T, GENG X, MA H D. Learning-driven interference-aware workload parallelization for streaming applications in heterogeneous cluster [J]. IEEE transactions on parallel and distributed systems, 2021, 32(1): 1–15. DOI: 10.1109/TPDS.2020.3008725
34	CHEN Z Y. Research on deep learning task scheduling based on small scale GPU cluster platform [M]. Hunan, China: National University of Defense Technology, 2019
35	QI Y F, He X. Dynamic priority task scheduling algorithm based on deep learning [J]. Computer systems and applications, 2023, 32(7): 195–201. DOI: 10.15888/j.cnki.csa.009169
36	KECKLER S W, DALLY W J, KHAILANY B, et al. GPUs and the future of parallel computing [J]. IEEE micro, 2011, 31(5): 7–17. DOI: 10.1109/MM.2011.89
37	AUGONNET C, THIBAULT S, NAMYST R, et al. StarPU: a unified platform for task scheduling on heterogeneous multicore architectures [M]//Lecture Notes in Computer Science. Berlin, Germany: Springer, 2009: 863–874. DOI: 10.1007/978-3-642-03869-3_80
38	ZHONG J L, HE B S. Kernelet: high-throughput GPU kernel executions with dynamic slicing and scheduling [J]. IEEE transactions on parallel and distributed systems, 2014, 25(6): 1522–1532. DOI: 10.1109/TPDS.2013.257
39	ZOU A, LI J, GILL C D, et al. RTGPU: real-time GPU scheduling of hard deadline parallel tasks with fine-grain utilization [J]. IEEE transactions on parallel and distributed systems, 2023, 34(5): 1450–1465. DOI: 10.1109/TPDS.2023.3235439
40	LÓPEZ‐ALBELDA B, LÁZARO‐MUÑOZ A J, GONZÁLEZ‐LINARES J M, et al. Heuristics for concurrent task scheduling on GPUs [J]. Concurrency and computation, 2020, 32(20): e5571. DOI: 10.1002/cpe.5571
41	HUANG Y H, GUO B, SHEN Y. GPU energy optimization based on task balance scheduling [J]. Journal of systems architecture, 2020, 107: 101808. DOI: 10.1016/j.sysarc.2020.101808
42	LI J, LIU L, WU Y, et al. Two-level task scheduling for irregular applications on GPU platform [J]. International journal of parallel programming, 2017, 45(1): 79–93. DOI: 10.1007/s10766-015-0387-0
43	KWON W, YU G-I, JEONG E, et al. Nimble: lightweight and parallel GPU task scheduling for deep learning [C]//The 34th International Conference on Neural Information Processing Systems. Curran Associates, 2020: 8343–8354. DOI: https://dl.acm.org/doi/10.5555/3495724.3496423
44	CHEN Y X, BROCK B, PORUMBESCU S, et al. Atos: a task-parallel GPU scheduler for graph analytics [C]//The 51st International Conference on Parallel Processing. ACM, 2022. DOI: 10.1145/3545008.3545056
45	TANG Z, DU L F, ZHANG X D, et al. AEML: an acceleration engine for multi-GPU load-balancing in distributed heterogeneous environment [J]. IEEE transactions on computers, 2022, 71(6): 1344–1357. DOI: 10.1109/TC.2021.3084407
46	LIMA J V F, GAUTIER T, DANJEAN V, et al. Design and analysis of scheduling strategies for multi-CPU and multi-GPU architectures [J]. Parallel computing, 2015, 44: 37–52. DOI: 10.1016/j.parco.2015.03.001
47	ALEBRAHIM S, AHMAD I. Task scheduling for heterogeneous computing systems [J]. The journal of supercomputing, 2017, 73(6): 2313–2338. DOI: 10.1007/s11227-016-1917-2

Task Scheduling Technology	Evaluation Metrics
Task Scheduling Technology	Speedup	Energy	Load balance	Makespan	Resource utilization	QoS
Hybrid map^[11]	√
Energy efficient task scheduling^[12]		√
Energy-minimized scheduling^[13]		√
Task scheduling with temperature constraint^[14–15]		√
PTA&WSLSA^[16]			√
Multi-granularity partition^[17]			√
Adaptive and transparent task scheduling^[18]			√
Dual approximation technique^[19]				√
Data partition^[20]				√
Large instance sheduling^[21]	√
Short task scheduling^[22]				√
Fine-grained scheduling^[23]					√
CNN-based task scheduling^[24]				√	√
GAS^[25]		√		√		√
Isolated scheduling^[26]	√				√	√
DeepBoot^[27]					√
QoS guarantee scheduling^[28]						√
Greedy heuristics^[29]			√
Local serach^[30]				√	√
CPU and GPU cooperative scheduling^[31]					√
Learning driven scheduling^[32–33]				√	√
Q-learning^[34]				√
Dynamic priority task scheduling^[35]				√		√
StarPU^[37]					√
Kernelet^[38]	√				√
RTGPU^[39]					√
Heuristics for concurrent task scheduling^[40]				√	√
Task balance scheduling^[41]		√
Two-level task Scheduling^[42]	√
Nimble^[43]	√
Atos^[44]	√
AEML^[45]			√		√

Task Scheduling Technology	Evaluation Metrics
Task Scheduling Technology	Speedup	Energy	Load balance	Makespan	Resource utilization	QoS
Hybrid map^[11]	√
Energy efficient task scheduling^[12]		√
Energy-minimized scheduling^[13]		√
Task scheduling with temperature constraint^[14–15]		√
PTA&WSLSA^[16]			√
Multi-granularity partition^[17]			√
Adaptive and transparent task scheduling^[18]			√
Dual approximation technique^[19]				√
Data partition^[20]				√
Large instance sheduling^[21]	√
Short task scheduling^[22]				√
Fine-grained scheduling^[23]					√
CNN-based task scheduling^[24]				√	√
GAS^[25]		√		√		√
Isolated scheduling^[26]	√				√	√
DeepBoot^[27]					√
QoS guarantee scheduling^[28]						√
Greedy heuristics^[29]			√
Local serach^[30]				√	√
CPU and GPU cooperative scheduling^[31]					√
Learning driven scheduling^[32–33]				√	√
Q-learning^[34]				√
Dynamic priority task scheduling^[35]				√		√
StarPU^[37]					√
Kernelet^[38]	√				√
RTGPU^[39]					√
Heuristics for concurrent task scheduling^[40]				√	√
Task balance scheduling^[41]		√
Two-level task Scheduling^[42]	√
Nimble^[43]	√
Atos^[44]	√
AEML^[45]			√		√

A Survey on Task Scheduling of CPU-GPU Heterogeneous Cluster

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 4

References 47

Related Articles 1

Recommended Articles

Metrics