A Survey on Machine Learning Based Proactive Caching

doi:10.12142/ZTECOM.201904007

ZTE Communications ›› 2019, Vol. 17 ›› Issue (4): 46-55.DOI: 10.12142/ZTECOM.201904007

• Special Topic • Previous Articles Next Articles

A Survey on Machine Learning Based Proactive Caching

Stephen ANOKYE^1,², Mohammed SEID^1,³, SUN Guolin¹()

^1.University of Electronic Science and Technology of China, Chengdu, Sichuan 611731, China
^2.University of Mines and Technology, Tarkwa 237, Ghana
^3.Dilla University, Dilla, Ethiopia

Received:2019-10-27 Online:2019-12-25 Published:2020-04-16
About author:Stephen ANOKYE received his M.Eng. degree in computer science from Hunan University, China in 2009 and his B.Sc. degree in computer science from Kwame Nkrumah University of Science and Technology, Ghana in 2004. He is currently a Ph.D. candidate at the University of Electronic Science and Technology of China. Between 2010 and 2012, he worked as a lecturer in the Department of Computer Science at Garden City University College in Ghana. Since 2012, he has become a lecturer at the Department of Computer Science and Engineering, the University of Mines and Technology, Ghana. His research interests are security in wireless sensor networks, mobile and cloud networks with AI, UAV networks, IoT, and 5G wireless networks.|Mohammed SEID received his B.Sc. and M.Sc. degrees in computer science from Ambo University, Ethiopia in 2010 and Addis Ababa University, Ethiopia in 2015, respectively. He is currently pursuing his Ph.D. degree in computer science and technology at University of Electronic Science and Technology of China. From 2010 to 2016, he worked in Dilla University, Ethiopia as a graduate assistant and lecturer. His interests include mobile edge computing, fog computing, UAV networks, IoT, and 5G wireless networks.|SUN Guolin (guolin.sun@uestc.edu.cn) received his B.S., M.S. and Ph.D. degrees all in communication and information system from University of Electronic Science and Technology of China (UESTC) in 2000, 2003 and 2005, respectively. After his Ph.D. graduation in 2005, he has got eight-year industrial work experience in wireless research and development for LTE, Wi-Fi, IoT, cognitive radio, localization and navigation. Before he joined UESTC as an associate professor in August 2012, he worked in Huawei Technologies Sweden. Dr. SUN has filed over 30 patents and published over 30 scientific conference and journal papers. He acted as TPC member of several conferences. Currently, he serves as a vice-chair of the 5G oriented cognitive radio SIG of the Technical Committee on Cognitive Networks (TCCN) of the IEEE Communication Society. His general research interests are software defined networks, network function virtualization, and radio resource management.

Abstract

Abstract:

The world today is experiencing an enormous increase in data traffic, coupled with demand for greater quality of experience (QoE) and performance. Increasing mobile traffic leads to congestion of backhaul networks. One promising solution to this problem is the mobile edge network (MEN) and consequently mobile edge caching. In this paper, a survey of mobile edge caching using machine learning is explored. Even though a lot of work and surveys have been conducted on mobile edge caching, our efforts in this paper are rather focused on the survey of machine learning based mobile edge caching. Issues affecting edge caching, such as caching entities, caching policies and caching algorithms, are discussed. The machine learning algorithms applied to edge caching are reviewed followed by a discussion on the challenges and future works in this field. This survey shows that edge caching can reduce delay and subsequently the backhaul traffic of the network; most caching is conducted at the small base stations (SBSs) and caching at unmanned aerial vehicles (UAVs) is recently used to accommodate mobile users who dissociate from SBSs. This survey also demonstrates that machine learning approach is the state of the art and reinforcement learning is predominant.

Key words: mobile edge caching, reinforcement learning, unmanned aerial vehicle

Stephen ANOKYE, Mohammed SEID, SUN Guolin. A Survey on Machine Learning Based Proactive Caching[J]. ZTE Communications, 2019, 17(4): 46-55.

Figures/Tables 4

References 61

1	IndexC.V.N. Cisco Visual Networking Index : Global Mobile Data Traffic Forecast, 2016 – 2021 [R]. CISCO, PapWhite. Feb. 2017
2	WANG N, HOSSAIN E, BHARGAVA V K. Backhauling 5G Small Cells: A Radio Resource Management Perspective [J]. IEEE Wireless Communications, 2015, 22(5): 41–49. DOI: 10.1109/MWC.2015.7306536 DOI
3	WANG S, ZHANG X, ZHANG Y, et al. A Survey on Mobile Edge Networks: Convergence of Computing, Caching and Communications [J]. IEEE Access, 2017, 5: 6757–6779. DOI: 10.1109/ACCESS.2017.2685434 DOI
4	WANG X, CHEN M, TALEB T, et al. Cache in the Air: Exploiting Content Caching and Delivery Techniques for 5G Systems [J]. IEEE Communications Magazine, 2014, 52(2): 131–139. DOI: 10.1109/MCOM.2014.6736753 DOI
5	SU Z, XU Q, HOU F, et al. Edge Caching for Layered Video Contents in Mobile Social Networks [J]. IEEE Transactions on Multimedia, 2017, 19(10): 2210–2221. DOI: 10.1109/TMM.2017.2733338 DOI
6	CHEN M, MOZAFFARI M, SAAD W, et al. Caching in the Sky: Proactive Deployment of Cache⁃Enabled Unmanned Aerial Vehicles for Optimized Quality⁃of⁃Experience [J]. IEEE Journal on Selected Areas in Communications, 2017, 35(5): 1046–1061. DOI: 10.1109/JSAC.2017.2680898 DOI
7	LYU J, ZENG Y, ZHANG R. UAV⁃Aided Offloading for Cellular Hotspot [J]. IEEE Transactions on Wireless Communications, 2018, 17(6): 3988–4001. DOI: 10.1109/TWC.2018.2818734 DOI
8	MERWADAY A, GUVENC I. UAV Assisted Heterogeneous Networks for Public Safety Communications [C]//IEEE Wireless Communications and Networking Conference Workshops (WCNCW), New Orleans, USA, 2015: 329–334. DOI: 10.1109/WCNCW.2015.7122576 DOI
9	MOZAFFARI M, SAAD W, BENNIS M, et al. Mobile Internet of Things: Can UAVs Provide an Energy⁃Efficient Mobile Architecture? [C]//IEEE Global Communications Conference, Washington DC, USA, 2016: 4–8. DOI: 10.1109/GLOCOM.2016.7841993 DOI
10	ZHAN C, ZENG Y, ZHANG R. Energy⁃Efficient Data Collection in UAV Enabled Wireless Sensor Network [J]. IEEE Wireless Communications. Letters, 2018, 7(3): 328–331. DOI: 10.1109/LWC.2017.2776922 DOI
11	SOORKI M N, MOZAFFARI M, SAAD W, et al. Resource Allocation for Machine⁃to⁃Machine Communications with Unmanned Aerial Vehicles [C]//IEEE Globecom Workshops, Washington, USA, 2016: 1–6. DOI: 10.1109/GLOCOMW.2016.7849026 DOI
12	GUPTA S, SHARMA T P. Cooperative Data Caching in MANETs and WSNs: A Survey[C]//International Conference on Intelligent Computing, Instrumentation and Control Technologies, Kannur, India, 2018: 6–7. DOI: 10.1109/ICICICT1.2017.8342787 DOI
13	ZHANG M, LUO H, ZHANG H. A Survey of Caching Mechanisms in Information⁃Centric Networking [J]. IEEE Communications Surveys & Tutorials, 2015, 17(3): 1473–1499. DOI: 10.1109/COMST.2015.2420097 DOI
14	IOANNOU A, WEBER S. A Survey of Caching Policies and Forwarding Mechanisms in Information⁃Centric Networking [J]. IEEE Communications Surveys & Tutorials, 2016, 18(4): 2847–2886. DOI: 10.1109/COMST.2016.2565541 DOI
15	ABOUAOMAR A, FILALI A, KOBBANE A. Caching, Device⁃to⁃Device and Fog Computing in 5th Cellular Networks Generation: Survey[C]//International Conference on Wireless Networks and Mobile Communications, Rabat, Morocco, 2017: 1–4. DOI: 10.1109/WINCOM.2017.8238174 DOI
16	GLASS S, MAHGOUB I, RATHOD M. Leveraging MANET⁃Based Cooperative Cache Discovery Techniques in VANETs: A Survey and Analysis [J]. IEEE Communications Surveys & Tutorials, 2017, 19(4): 2640–2661. DOI: 10.1109/COMST.2017.2707926 DOI
17	DIN I U, HASSAN S, KHAN M K, et al. Caching in Information⁃Centric Networking: Strategies, Challenges, and Future Research Directions [J]. IEEE Communications Surveys & Tutorials, 2018, 20(2): 1443–1474. DOI: 10.1109/COMST.2017.2787609 DOI
18	LI L, ZHAO G, BLUM R S. A Survey of Caching Techniques in Cellular Networks: Research Issues and Challenges in Content Placement and Delivery Strategies [J]. IEEE Communications Surveys & Tutorials, 2018, 20(3): 1710–1732. DOI: 10.1109/COMST.2018.2820021 DOI
19	PARVEZ I, RAHMATI A, GUVENC I, et al. A Survey on Low Latency Towards 5G: RAN, Core Network and Caching Solutions [J]. IEEE Commun. Surv. Tutorials, 2018, 20(4): 3098–3130. DOI: 10.1109/COMST.2018.2841349 DOI
20	GOIAN H S, AL-JARRAH O Y, MUHAIDAT S, et al. Popularity⁃Based Video Caching Techniques for Cache⁃Enabled Networks: A Survey [J]. IEEE Access. 2019, 7: 27699–27719. DOI: 10.1109/ACCESS.2019.2898734 DOI
21	CAO H, CAI J. Context⁃Aware Proactive Caching for Heterogeneous Networks with Energy Harvesting: An Online Learning Approach [C]//IEEE International Congress on Cybermatics, Halifax, Canada, 2018. DOI: 10.1109/Cybermatics_2018.2018.00073 DOI
22	ZHONG C, GURSOY M C, VELIPASALAR S. A Deep Reinforcement Learning⁃Based Framework for Content Caching [C]//52nd Annual Conference on Information Sciences and Systems, Princeton, USA. 2018. DOI: 10.1109/CISS.2018.8362276
23	TSAI K C, WANG L, HAN Z. Mobile Social Media Networks Caching with Convolutional Neural Network [C]//IEEE Wireless Communications and Networking Conference Workshops (WCNCW), Barcelona, Spain, 2018: 83–88. DOI: 10.1109/WCNCW.2018.8368988 DOI
24	BLASCO P, GUNDUZ D. Learning⁃Based Optimization of Cache Content in a Small Cell Base Station [C]//IEEE International Conference on Communications (ICC), Sydney, Australia, 2014: 1897–1903. DOI: 10.1109/ICC.2014.6883600 DOI
25	IM Y, PRAHLADAN P, KIM T H, et al. SNN⁃Cache: A Practical Machine Learning⁃Based Caching System Utilizing the Inter⁃Relationships of Requests [C]//52nd Annual Conference on Information Sciences and Systems, Princeton, USA, 2018. DOI: 10.1109/CISS.2018.8362281 DOI
26	JIANG W, FENG G, QIN S, et al. Multi⁃Agent Reinforcement Learning for Efficient Content Caching in Mobile D2D Networks [J]. IEEE Transactions on Wireless Communications, 2019, 18(3): 1610–1622. DOI: 10.1109/TWC.2019.2894403 DOI
27	LI Y, ZHONG C, GURSOY M C, et al. Learning⁃Based Delay⁃Aware Caching in Wireless D2D Caching Networks [J]. IEEE Access, 2018, 6: 77250–77264. DOI: 10.1109/ACCESS.2018.2881038 DOI
28	CHEN B, YANG C. Caching Policy for Cache⁃Enabled D2D Communications by Learning User Preference [J]. IEEE Transactions on Communications, 2018, 66(12): 6586–6601. DOI: 10.1109/TCOMM.2018.2863364 DOI
29	WANG C, WANG S, LI D, et al. Q⁃Learning Based Edge Caching Optimization for D2D Enabled Hierarchical Wireless Networks [C]//IEEE 15th International Conference on Mobile Ad Hoc and Sensor Systems (MASS), Chengdu, China, 2018: 55–63. DOI: 10.1109/MASS.2018.00019 DOI
30	CHEN M, SAAD W, YIN C. Liquid State Machine Learning for Resource Allocation in a Network of Cache⁃Enabled LTE⁃U UAVs [C]//IEEE Global Communications Conference, Singapore, Singapore, 2018: 1–6. DOI: 10.1109/GLOCOM.2017.8254746 DOI
31	PASCHOS G, BASTUG E, LAND I, et al. Wireless Caching: Technical Misconceptions and Business Barriers [J]. IEEE Communications Magazine, 2016, 54(8): 16–22. DOI: 10.1109/MCOM.2016.7537172 DOI
32	LIU W X, ZHANG J, LIANG Z W, et al. Content Popularity Prediction and Caching for ICN: A Deep Learning Approach with SDN [J]. IEEE Access, 2017, 6: 5075–5089. DOI: 10.1109/ACCESS.2017.2781716 DOI
33	BRESLAU L, CAO P, FAN L, et al. Web Caching and Zipf⁃Like Distributions: Evidence and Implications [C]//IEEE INFOCOM, New York, USA, 1999: 126–134. DOI: 10.1109/INFCOM.1999.749260 DOI
34	AHLEHAGH H, DEY S. Video⁃Aware Scheduling and Caching in the Radio Access Network [J]. IEEE/ACM Transactions on Networking, 2014, 22(5): 1444–1462. DOI: 10.1109/TNET.2013.2294111 DOI
35	SENGUPTA A, AMURU S, TANDON R, et al. Learning Distributed Caching Strategies in Small Cell Networks [C]//11th International Symposium on Wireless Communications Systems, Barcelona, Spain, 2014: 917–921. DOI: 10.1109/ISWCS.2014.6933484 DOI
36	GU J, WANG W, HUANG A, et al. Distributed Cache Replacement for Caching⁃Enable Base Stations in Cellular Networks [C]//IEEE International Conference on Communications (ICC), Sydney, Australia, 2014: 2648–265. DOI: 10.1109/ICC.2014.6883723 DOI
37	SUNG J, KIM K, KIM J, et al. Efficient Content Replacement in Wireless Content Delivery Network with Cooperative Caching[C]//IEEE International Conference on Machine Learning and Applications, Anaheim, USA, 2016. DOI: 10.1109/ICMLA.2016.0096 DOI
38	PANG H, LIU J, FAN X, et al. Toward Smart and Cooperative Edge Caching for 5G Networks: A Deep Learning Based Approach[C]//IEEE/ACM 26th International Symposium on Quality of Service, Banff, Canada, 2018: 1–6, 2019. DOI: 10.1109/IWQoS.2018.8624176 DOI
39	NIU Y, QIN X, ZHANG Z. A Learning⁃Based Cooperative Caching Strategy in D2D Assisted Cellular Networks [C]//24th Asia⁃Pacific Conference on Communications, Ningbo, China, 2018: 269–274. DOI: 10.1109/APCC.2018.8633483 DOI
40	SAPUTRA Y M, HOANG D T, NGUYEN D N, et al. Distributed Deep Learning at the Edge: A Novel Proactive and Cooperative Caching Framework for Mobile Edge Networks [J]. IEEE Wireless Communications Letters, vol. 8, no. 4, pp. 1220–1223, 2019. DOI: 10.1109/LWC.2019.2912365 DOI
41	THAR K, OO T Z, TUN Y K, et al. A Deep Learning Model Generation Framework for Virtualized Multi⁃Access Edge Cache Management [J]. IEEE Access, 2019, 7: 62734–62749. DOI: 10.1109/ACCESS.2019.2916080 DOI
42	PAHL M O, LIEBALD S, WUSTRICH L. Machine⁃Learning Based IoT Data Caching [C]//IFIP/IEEE Symposium on Integrated Network and Service Management, Arlington, USA, 2019
43	WEI Y, YU F R, SONG M, et al. Joint Optimization of Caching, Computing, and Radio Resources for Fog⁃Enabled IoT Using Natural Actor-Critic Deep Reinforcement Learning [J]. IEEE Internet of Things Journal, 2019, 6(2): 2061–2073. DOI: 10.1109/JIOT.2018.2878435 DOI
44	GUO K, YANG C. Temporal⁃Spatial Recommendation for Caching at Base Stations via Deep Reinforcement Learning [J]. IEEE Access, 2019, 7:8519–58532. DOI: 10.1109/ACCESS.2019.2914500 DOI
45	TAN L T, HU R Q. Mobility⁃Aware Edge Caching and Computing in Vehicle Networks: A Deep Reinforcement Learning [J]. IEEE Transactions on Vehicular Technology, 2018, 67(11): 10190–10203. DOI: 10.1109/TVT.2018.2867191 DOI
46	ZHOU Y, PENG M, YAN S, et al. Deep Reinforcement Learning Based Coded Caching Scheme in Fog Radio Access Networks [C]//IEEE/CIC International Conference on Communications in China, Beijing, China, 2018: 309–313. DOI: 10.1109/ICCChinaW.2018.8674478 DOI
47	KUMAR N, SWAIN S N, SIVA RAM MURTHY C. A Novel Distributed Q⁃Learning Based Resource Reservation Framework for Facilitating D2D Content Access Requests in LTE⁃A Networks [J]. IEEE Transactions on Network and Service Management, 2018, 15(2): 718–731. DOI: 10.1109/TNSM.2018.2807594 DOI
48	VARANASI V S, CHILUKURI S. Adaptive Differentiated Edge Caching with Machine Learning for V2X Communication[C]//International Conference on Communication Systems and Networks, Bengaluru, India, 2019.DOI: 10.1109/COMSNETS.2019.8711328 DOI
49	SHEN G, LI P, PAN Z W, et al. Machine learning based small cell cache strategy for ultra dense networks[C]//9th International Conference on Wireless Communications and Signal Processing, Nanjing, China, 2017: 1–6. DOI: 10.1109/WCSP.2017.8170936 DOI
50	LEI L, YOU L, DAI G, et al. A Deep Learning Approach for Optimizing Content Delivering in Cache⁃Enabled HetNet [C]//International Symposium on Wireless Communication Systems, Bologna, Italy, 2017: 449–453. DOI: 10.1109/ISWCS.2017.8108157 DOI
51	HE Y, YU F R, ZHAO N, et al. Secure Social Networks in 5G Systems with Mobile Edge Computing, Caching, and Device⁃to⁃Device Communications [J]. IEEE Wireless Communications, 2018, 25(3): 103–109. DOI: 10.1109/MWC.2018.1700274 DOI
52	CHENG P, MA C, DING M, et al. Localized Small Cell Caching: A Machine Learning Approach Based on Rating Data [J]. IEEE Transactions on Communications, 2019, 67(2): 1663–1676. DOI: 10.1109/TCOMM.2018.2878231 DOI
53	SUN G, AL⁃WARD H, BOATENG G O, et al. Autonomous Cache Resource Slicing and Content Placement at Virtualized Mobile Edge Network [J]. IEEE Access, 2019, 7(c): 84727–84743. DOI: 10.1109/ACCESS.2019.2923021 DOI
54	QIU C, YAO H, YU F R, et al. Deep Q⁃Learning Aided Networking, Caching, and Computing Resources Allocation in Software⁃Defined Satellite⁃Terrestrial Networks [J]. IEEE Transactions on Vehicular Technology, 2019, 68(6): 5871–5883. DOI: 10.1109/TVT.2019.2907682 DOI
55	XU X, ZENG Y, GUAN Y L, et al. Overcoming Endurance Issue: UAV⁃Enabled Communications with Proactive Caching [J]. IEEE Journal on Selected Areas in Communications, 2018, 36(6): 1231–1244. DOI: 10.1109/JSAC.2018.2844979 DOI
56	JIANG B, YANG J, XU H, et al. Multimedia Data Throughput Maximization in Internet⁃of⁃Things System Based on Optimization of Cache-Enabled UAV [J]. IEEE Internet of Things Journal, 2019, 6(2): 3525–3532. DOI: 10.1109/JIOT.2018.2886964 DOI
57	SADEGHI A, SHEIKHOLESLAMI F, GIANNAKIS G B. Optimal and Scalable Caching for 5G Using Reinforcement Learning of Space⁃Time Popularities [J]. IEEE Journal of Selected Topics in Signal Processing, 2018, 12(1): 180–190. DOI: 10.1109/JSTSP.2017.2787979 DOI
58	JIANG W, FENG G, QIN S, et al. Multi⁃Agent Reinforcement Learning Based Cooperative Content Caching for Mobile Edge Networks [J]. IEEE Access, 2019, 7: 61856–61867. DOI: 10.1109/ACCESS.2019.2916314 DOI
59	ZHU H, CAO Y, WEI X, et al. Caching Transient Data for Internet of Things: A Deep Reinforcement Learning Approach [J]. IEEE Internet of Things Journal, 2019, 6(2): 2074–2083. DOI: 10.1109/JIOT.2018.2882583 DOI
60	HE Y, ZHAO N, YIN H. Integrated Networking, Caching, and Computing for Connected Vehicles: A Deep Reinforcement Learning Approach [J]. IEEE Transactions on Vehicular Technology, 2018, 67(1): 44–55. DOI: 10.1109/TVT.2017.2760281 DOI
61	HE Y, RICHARD Y F, ZHAO N, et al. Software⁃Defined Networks with Mobile Edge Computing and Caching for Smart Cities: A Big Data Deep Reinforcement Learning Approach [J]. IEEE Communications Magazine, 2017, 55(12): 31–37. DOI: 10.1109/MCOM.2017.1700246 DOI

Work Area	Literature	Key Points
Caching entities	[15], [21], [23], [31], [37], [47], [49], [51]–[53]	Macro base station (MBS)
	[18], [21]–[25], [28], [29], [31], [35]–[38], [41], [43], [44], [46], [49], [50], [52]–[54]	Small base station (SBS)
	[25]–[29], [39], [47], [51], [55]	Device
	[6], [30], [55], [56]	Unmanned aerial vehicle (UAV)
Content popularity	[31]	Static model
Content popularity	[6], [21], [22], [24], [26]–[29], [32], [34]–[36], [39]–[41], [44], [45], [49], [50]–[52], [54]–[57]	Dynamic model
Caching policies and algorithms	[16], [34], [48]	Conventional caching policies
	[28], [29], [34], [44], [52], [58]	User preference based polices
	[6], [21]–[30], [35]–[41], [43]–[47], [49]–[51], [53]–[57]	Learning based policies
	[34]	Non-cooperative policies
	[22], [23], [26], [28], [30], [37]–[40], [51], [54], [55], [57]	Cooperative policies
Caching file types	[6], [21]–[24], [26]–[30], [32], [34]–[41], [44]–[47], [49]–[51], [53], [55], [57]–[59]	Multimedia data
Caching file types	[42], [43], [48], [59]	Internet of Things (IoT) data
Mobility awareness	[6], [29], [30], [44], [45], [48], [49], [51], [52]	Spatial and temporal properties of user mobility
Problem	[23], [24], [26], [27], [34], [37]–[39], [41], [43], [46], [50], [52], [55], [58]	Delay
	[22]–[26], [37], [44]	Hit rate
	[35], [36], [39]–[41], [45]–[47]	Backhaul/fronthaul
	[6], [34], [49], [58],	Quality of experience (QoE)

Work Area	Literature	Key Points
Caching entities	[15], [21], [23], [31], [37], [47], [49], [51]–[53]	Macro base station (MBS)
	[18], [21]–[25], [28], [29], [31], [35]–[38], [41], [43], [44], [46], [49], [50], [52]–[54]	Small base station (SBS)
	[25]–[29], [39], [47], [51], [55]	Device
	[6], [30], [55], [56]	Unmanned aerial vehicle (UAV)
Content popularity	[31]	Static model
Content popularity	[6], [21], [22], [24], [26]–[29], [32], [34]–[36], [39]–[41], [44], [45], [49], [50]–[52], [54]–[57]	Dynamic model
Caching policies and algorithms	[16], [34], [48]	Conventional caching policies
	[28], [29], [34], [44], [52], [58]	User preference based polices
	[6], [21]–[30], [35]–[41], [43]–[47], [49]–[51], [53]–[57]	Learning based policies
	[34]	Non-cooperative policies
	[22], [23], [26], [28], [30], [37]–[40], [51], [54], [55], [57]	Cooperative policies
Caching file types	[6], [21]–[24], [26]–[30], [32], [34]–[41], [44]–[47], [49]–[51], [53], [55], [57]–[59]	Multimedia data
Caching file types	[42], [43], [48], [59]	Internet of Things (IoT) data
Mobility awareness	[6], [29], [30], [44], [45], [48], [49], [51], [52]	Spatial and temporal properties of user mobility
Problem	[23], [24], [26], [27], [34], [37]–[39], [41], [43], [46], [50], [52], [55], [58]	Delay
	[22]–[26], [37], [44]	Hit rate
	[35], [36], [39]–[41], [45]–[47]	Backhaul/fronthaul
	[6], [34], [49], [58],	Quality of experience (QoE)

Type of Machine Learning	Literature	Algorithm
Supervised learning	[6]	Eco state network
	[30]	Liquid state network
	[22],[49]	K-Nearest Neighbour (KNN)
	[48]	Kernel ridge regression (KRR)
	[38],[40],[50]	Deep learning
	[23]	Convolutional neural network (CNN)
	[25]	Stimulable neural network (SNN)
Unsupervised learning	[27],[28],[32],[49]	K-means
	[27],[28]	Greedy based algorithm
	[32]	Stacked auto encoders (SAEs) deep learning
Reinforcement learning	[57]	Q-learning
	[44]–[46],[51]	Deep Q-learning
	[60]	Double-dueling-deep Q-network
	[22],[34],[37],[43]	Actor critic
	[26],[58]	Multi agent Q-learning
	[21]	Post decision state based approximate RL (PDS-ARL)
	[52]	Discrete learning automata (DLA)

Type of Machine Learning	Literature	Algorithm
Supervised learning	[6]	Eco state network
	[30]	Liquid state network
	[22],[49]	K-Nearest Neighbour (KNN)
	[48]	Kernel ridge regression (KRR)
	[38],[40],[50]	Deep learning
	[23]	Convolutional neural network (CNN)
	[25]	Stimulable neural network (SNN)
Unsupervised learning	[27],[28],[32],[49]	K-means
	[27],[28]	Greedy based algorithm
	[32]	Stacked auto encoders (SAEs) deep learning
Reinforcement learning	[57]	Q-learning
	[44]–[46],[51]	Deep Q-learning
	[60]	Double-dueling-deep Q-network
	[22],[34],[37],[43]	Actor critic
	[26],[58]	Multi agent Q-learning
	[21]	Post decision state based approximate RL (PDS-ARL)
	[52]	Discrete learning automata (DLA)

[1]	SHEN Jiahao, JIANG Ke, TAN Xiaoyang. Boundary Data Augmentation for Offline Reinforcement Learning [J]. ZTE Communications, 2023, 21(3): 29-36.
[2]	REN Min, XU Renyu, ZHU Ting. Double Deep Q-Network Decoder Based on EEG Brain-Computer Interface [J]. ZTE Communications, 2023, 21(3): 3-10.
[3]	FENG Bingyi, FENG Mingxiao, WANG Minrui, ZHOU Wengang, LI Houqiang. Multi-Agent Hierarchical Graph Attention Reinforcement Learning for Grid-Aware Energy Management [J]. ZTE Communications, 2023, 21(3): 11-21.
[4]	YU Junpeng, CHEN Yiyu. A Practical Reinforcement Learning Framework for Automatic Radar Detection [J]. ZTE Communications, 2023, 21(3): 22-28.
[5]	YOU Qian, XU Qian, YANG Xin, ZHANG Tao, CHEN Ming. RIS-Assisted UAV-D2D Communications Exploiting Deep Reinforcement Learning [J]. ZTE Communications, 2023, 21(2): 61-69.
[6]	LI Yuting, DING Yi, GAO Jiangchuan, LIU Yusha, HU Jie, YANG Kun. UAV Autonomous Navigation for Wireless Powered Data Collection with Onboard Deep Q-Network [J]. ZTE Communications, 2023, 21(2): 80-87.
[7]	JIA Haonan, HE Zhenqing, TAN Wanlong, RUI Hua, LIN Wei. Distributed Multi-Cell Multi-User MISO Downlink Beamforming via Deep Reinforcement Learning [J]. ZTE Communications, 2022, 20(4): 69-77.
[8]	LI Xiuxian, LI Zhetao, OUYANG Yan, DUAN Haohua, XIANG Liyao. Using UAV to Detect Truth for Clean Data Collection in Sensor‑Cloud Systems [J]. ZTE Communications, 2021, 19(3): 30-45.
[9]	JI Hong, ZHANG Tianxiang, ZHANG Kai, WANG Wanyuan, WU Weiwei. Efficient Network Slicing with Dynamic Resource Allocation [J]. ZTE Communications, 2021, 19(1): 11-19.
[10]	LIN Xinhua, ZHANG Jing, LI Qiang. Cluster Head Selection Algorithm for UAV Assisted Clustered IoT Network Utilizing Blockchain [J]. ZTE Communications, 2021, 19(1): 30-38.
[11]	Mohammed SEID, Stephen ANOKYE, SUN Guolin. Machine Learning Based Unmanned Aerial Vehicle Enabled Fog-Radio Aerial Vehicle Enabled Fog-Radio Access Network and Edge Computing [J]. ZTE Communications, 2019, 17(4): 33-45.
[12]	DONG Shaokang, CHEN Jiarui, LIU Yong, BAO Tianyi, GAO Yang. Reinforcement Learning from Algorithm Model to Industry Innovation: A Foundation Stone of Future Artificial Intelligence [J]. ZTE Communications, 2019, 17(3): 31-41.
[13]	LI Tongxin, SHENG Min, LYU Ruiling, LIU Junyu, LI Jiandong. UAV Assisted Heterogeneous Wireless Networks: Potentials and Challenges [J]. ZTE Communications, 2018, 16(2): 3-8.

A Survey on Machine Learning Based Proactive Caching

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 4

References 61

Related Articles 13

Recommended Articles

Metrics