Indoor Environment and Human Sensing via Millimeter Wave Radio: A Review

doi:10.12142/ZTECOM.202103004

ZTE Communications ›› 2021, Vol. 19 ›› Issue (3): 22-29.doi: 10.12142/ZTECOM.202103004

• Special Topic • Previous Articles Next Articles

Indoor Environment and Human Sensing via Millimeter Wave Radio: A Review

LIU Haipeng(), ZHANG Xingyue, ZHOU Anfu, LIU Liang, MA Huadong

Beijing University of Posts and Telecommunications, Beijing 100876, China

Received:2021-06-10 Online:2021-09-25 Published:2021-10-11
About author:LIU Haipeng (fengxudi@qq.com) is a Ph.D. student of Beijing University of Posts and Telecommunications, China. He is directly recommended for a doctorate degree from undergraduate. His research interest lies in wireless networks and mobile sensing, with the emphasis on ubiquitous mobile sensing for enabling new Internet-of-Things applications.|ZHANG Xingyue is a second-grade student at the International School of Beijing University of Posts and Telecommunications, China. She is currently majoring in telecommunications engineering and management, and is about to enter the direction of multimedia learning.|ZHOU Anfu received his Ph.D. degree in computer science from the Institute of Computing Technology, Chinese Academy of Science, China in 2012. Before that, he received B.S. degree from the Renmin University of China. He is now a professor at the School of Computer Science, Beijing University of Posts and Telecommunications, China. His research interest lies in mobile computing and wireless networking and IoT systems.|LIU Liang obtained his Ph.D. degree from Beijing University of Posts and Telecommunications, China in 2009, and conducted visiting research in TAMU in the US. His main research directions are multimedia and sensor networks. He has published nearly 40 papers in international authoritative journals and conferences such as IEEE Transactions and INFOCOM.|MA Huadong is an IEEE Fellow. He received his Ph.D. degree in computer science from the Institute of Computing Technology, Chinese Academy of Science, China in 1995. He is now a Chang Jiang Scholar Professor and Director of Beijing Key Laboratory of Intelligent Telecommunications Software and Multimedia. He is also the Executive Dean of School of Computer Science, Beijing University of Posts and Telecommunications, China. His research interests include multimedia systems and networking, sensor networks, and the Internet of Things. He has authored over 200 papers and four books.
Supported by:
the National Key R&D Program of China(2019YFB2102202);the A3 Foresight Program of NSFC(62061146002);the NSFC(61772084);the Funds for Creative Research Groups of China(61921003);the Youth Top Talent Support Program, the 111 Project(B18008);the Fundamental Research Funds for the Central Universities(2019XD-A13)

Abstract

Abstract:

With the rapid development of 5G technology, more and more attention has been attracted to mmWave sensing. As an emerging sensing medium, mmWave has the advantages of both high sensitivity and precision. Different from its networking applications, the core method of mmWave sensing is to analyze the reflected signal changes containing the relevant information of different surrounding environments. In this paper, we conduct a systemic review for mmWave sensing. We first summarize the prior works on environmental sensing with different signal analysis methods. Then, we classify and discuss the work of sensing humans, including their behavior and gestures. Finally, we discuss and put forward more possibilities of mmWave human perception.

Key words: millimeter wave radio, review, environment construction, human sensing, 5G

LIU Haipeng, ZHANG Xingyue, ZHOU Anfu, LIU Liang, MA Huadong. Indoor Environment and Human Sensing via Millimeter Wave Radio: A Review[J]. ZTE Communications, 2021, 19(3): 22-29.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

Figures/Tables 7

Figure 1

Table 1

Figure 2

Table 2

Figure 3

Figure 4

Table 3

References 23

1	ZHU Y Z, ZHU Y B, ZHAO B Y, et al. Reusing 60 GHz radios for mobile radar imaging [C]//The 21st Annual International Conference on Mobile Computing and Networking. Paris, France: ACM, 2015: 103–116. DOI: 10.1145/2789168.2790112 doi: 10.1145/2789168.2790112
2	PALACIOS J, CASARI P, WIDMER J. JADE: Zero‑knowledge device localization and environment mapping for millimeter wave systems [C]//IEEE Conference on Computer Communications. Atlanta, USA: IEEE, 2017: 1–9. DOI: 10.1109/INFOCOM.2017.8057183 doi: 10.1109/INFOCOM.2017.8057183
3	MENG Z, FU S, YAN J, et al. Gait recognition for co‑existing multiple people using millimeter wave sensing [C]//The AAAI Conference on Artificial Intelligence. New York, USA: AAAI, 2020, 34(01): 849–856. DOI: index.php/AAAI/article/view/5430 doi: index.php/AAAI/article/view/5430
4	WANG S W, SONG J, LIEN J, et al. Interacting with soli: exploring fine‑grained dynamic gesture recognition in the radio‑frequency spectrum [C]//The 29th Annual Symposium on User Interface Software and Technology. Tokyo, Japan: ACM, 2016: 851–860. DOI: 10.1145/2984511.2984565 doi: 10.1145/2984511.2984565
5	LIEN J, GILLIAN N, KARAGOZLER M E, et al. Soli: ubiquitous gesture sensing with millimeter wave radar [J]. ACM transactions on graphics, 2016, 35(4): 1–19. DOI: 10.1145/2897824.2925953 doi: 10.1145/2897824.2925953
6	WEI T, ZHANG X Y. MTrack: high‑precision passive tracking using millimeter wave radios [C]//The 21st Annual International Conference on Mobile Computing and Networking. Paris, France: ACM, 2015: 117–129. DOI: 10.1145/2789168.2790113 doi: 10.1145/2789168.2790113
7	PALACIOS J, BIELSA G, CASARI P, et al. Communication‑driven localization and mapping for millimeter wave networks [C]//IEEE Conference on Computer Communications. Honolulu, USA: IEEE, 2018: 2402–2410. DOI: 10.1109/INFOCOM.2018.8485819 doi: 10.1109/INFOCOM.2018.8485819
8	WEI T, ZHOU A, ZHANG X. Facilitating robust 60 GHz network deployment by sensing ambient reflectors [C]//14th USENIX Symposium on Networked Systems Design and Implementation. Boston, USA: IEEE, 2017: 213–226
9	ZHOU A F, ZHANG X Y, MA H D. Beam‑forecast: facilitating mobile 60 GHz networks via model‑driven beam steering [C]//IEEE Conference on Computer Communications. Atlanta, USA: IEEE, 2017: 1–9. DOI:10.1109/INFOCOM.2017.8057188 doi: 10.1109/INFOCOM.2017.8057188
10	ZHU Y, YAO Y, ZHAO B Y, et al. Object recognition and navigation using a single networking device [C]//Annual International Conference on Mobile Systems, Applications, and Services. Niagara Falls, USA: ACM, 2017: 265–277. DOI: 10.1145/3081333.3081339 doi: 10.1145/3081333.3081339
11	YEO H S, FLAMICH G, SCHREMPF P, et al. RadarCat: radar categorization for input & interaction [C]//The 29th Annual Symposium on User Interface Software and Technology. Tokyo, Japan: ACM, 2016: 833–841.DOI: 10.1145/2984511.2984515 doi: 10.1145/2984511.2984515
12	ZHOU A F, YANG S Y, YANG Y, et al. Autonomous environment mapping using commodity millimeter‑wave network device [C]//IEEE Conference on Computer Communications. Paris, France: IEEE, 2019: 1126–1134. DOI:10.1109/INFOCOM.2019.8737624 doi: 10.1109/INFOCOM.2019.8737624
13	ZHOU A F, XU S Q, WANG S, et al. Robot navigation in radio beam space: Leveraging robotic intelligence for seamless mmWave network coverage [C]//The 20th ACM International Symposium on Mobile Ad Hoc Networking and Computing. Catania, Italy: ACM, 2019: 161–170. DOI: 10.1145/3323679.3326514 doi: 10.1145/3323679.3326514
14	YANG Z C, PATHAK P H, ZENG Y Z, et al. Monitoring vital signs using millimeter wave [C]//Proceedings of the 17th ACM International Symposium on Mobile Ad Hoc Networking and Computing. Paderborn, Germany: ACM, 2016: 211–220. DOI: 10.1145/2942358.2942381 doi: 10.1145/2942358.2942381
15	GU T B, FANG Z, YANG Z C, et al. MmSense: multi‑person detection and identification via mmWave sensing [C]//The 3rd ACM Workshop on Millimeter‑wave Networks and Sensing Systems. Los Cabos, Mexico: ACM, 2019: 45–50. DOI: 10.1145/3349624.3356765 doi: 10.1145/3349624.3356765
16	LIU H P, BAI X W, GAO H Y, et al. MID: accurate and robust user identification and authentication through hand‑gesture sensing with mmwave radar [J]. Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies, 2021.
17	SMITH K A, CSECH C, MURDOCH D, et al. Gesture recognition using mm‑wave sensor for human‑car interface [J]. IEEE sensors letters, 2018, 2(2): 1–4. DOI: 10.1109/LSENS.2018.2810093 doi: 10.1109/LSENS.2018.2810093
18	PATRA A, GEUER P, MUNARI A, et al. Mm‑wave radar based gesture recognition: development and evaluation of a low‑power, low‑complexity system [C]//The 2nd ACM Workshop on Millimeter Wave Networks and Sensing Systems. New Delhi, India: ACM, 2018: 51–56. DOI: 10.1145/3264492.3264501 doi: 10.1145/3264492.3264501
19	LIU H P, WANG Y H, ZHOU A F, et al. Real‑time arm gesture recognition in smart home scenarios via millimeter wave sensing [J]. Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies, 2020, 4(4): 140. DOI: 10.1145/3432235 doi: 10.1145/3432235
20	PALIPANA S, SALAMI D, LEIVA L A, et al. Pantomime: mid‑air gesture recognition with sparse millimeter‑wave radar point clouds [J]. Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies, 2021, 5(1): 1–27. DOI:10.1145/3448110 doi: 10.1145/3448110
21	SANTHALINGAM P S, HOSAIN A A, ZHANG D, et al. MmASL: environment‑independent asl gesture recognition using 60 GHz millimeter‑wave signals [J]. Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies, 2020, 4(1): 1–30. DOI: 10.1145/3381010 doi: 10.1145/3381010
22	LIU H P, CUI K N, HU K Y, et al. Environment‑independent mmWave sensing based gesture recognition via transfer learning [J]. Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies, 2021
23	XUE H, JU Y, MIAO C, et al. MmMesh: towards 3D real‑time dynamic human mesh construction using millimeter‑wave [C]//Proceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services. 2021: 269–282. DOI: 10.1145/3458864.3467679 doi: 10.1145/3458864.3467679

Citation	Usage	Technology	Method
JADE	Position the client	An iterative algorithm	Both phase and RSS
CLAM	Map the environment	A distributed localization algorithm	Both phase and RSS
E-Mi	Boost the indoor network	A multi-path analysis framework	Both phase and RSS
Beam-Forcast	Improve the mobile links	Reverse engineering of E-Mi	Both phase and RSS
RSA	Position object & identify materials	Move Rx along trajectory while collecting	Only RSS
Ulysses	Image environment	Bind Tx & Rx together to collect reflection signals	Only RSS
RadarCat	Identify plenty of materials	Classification by learning on the radar signals	Only RSS
mmRanger	Sense environment	Automatically collect signals	RSS with a robot
miDroid	Improve mobile links	Set a network relay piggybacked on the robot	RSS with a robot

Citation	Usage	Equipment	Technology	Whether AI
Deep-soli	Gesture recognition	Soli	Range-Doppler images	Customized CNN
Ubi-soli	Gesture recognition	Soli	Multiple abstract representations	Random forest
HCI	Vehicular gesture recognition	Soli	Several physical features	Random forest
MengZhenGait	Identify different users	TI-IWR1443&6843	Three spatial features	MmGaitNet
MmVital	Monitor vital signs	Horn antennas	Extract periodic changes	No
MTrack	Track a vertical finger	Horn antennas	Combine target’s angle and phase change	No
MmSense	Multi-human detection	Free	Features of 60 GHz signal	No
MID	Gesture recognition	Free	MmWave sensing	No
LowCostGes	Detect gesture	RIC60a	Extract power profile and AoA	No
MmASL	Home-assistant system	Free	Features of 60 GHz signal	No
MHomeGes	Smart home-usage	TI-IWR1443	MmWave sensing	No
MTranseSee	User recognition	Free	MmWave radar	No
Pantomime	Gesture recognition	Pantomime	MmWave sensing	No
MmMesh	Human mesh construction	Free	Deep learning framework	No

	AI	Non-AI
Human dynamics	2,3,5,6	1,7,16,20,21,22,23
Indoor environment	8	9,11,12,13,14,15,17,18,19

[1]	YAN Xincheng, TENG Huiyun, PING Li, JIANG Zhihong, ZHOU Na. Study on Security of 5G and Satellite Converged Communication Network [J]. ZTE Communications, 2021, 19(4): 79-89.
[2]	XIAO Kai, LIU Xing, HAN Xianghui, HAO Peng, ZHANG Junfeng, ZHOU Dong, WEI Xingguang. Flexible Multiplexing Mechanism for Coexistence of URLLC and EMBB Services in 5G Networks [J]. ZTE Communications, 2021, 19(2): 82-90.
[3]	LIU Zhuang, GAO Yin, LI Dapeng, CHEN Jiajun, HAN Jiren. Enabling Energy Efficiency in 5G Network [J]. ZTE Communications, 2021, 19(1): 20-29.
[4]	ZHANG Jing, WEI Xiao, CHENG Junfeng, FENG Xu. Satellite E2E Network Slicing Based on 5G Technology [J]. ZTE Communications, 2020, 18(4): 26-33.
[5]	LI Yezhen, REN Yongli, YANG Fan, XU Shenheng, ZHANG Jiannian. A Novel 28 GHz Phased Array Antenna for 5G Mobile Communications [J]. ZTE Communications, 2020, 18(3): 20-25.
[6]	CHANG Su-Wei, LIN Chueh-Jen, TSAI Wen-Tsai, HUNG Tzu-Chieh, HUANG Po-Chia. Design of Millimeter-Wave Antenna-in-Package (AiP) for 5G NR [J]. ZTE Communications, 2020, 18(3): 26-32.
[7]	HE Yejun, JIANG Jiachun, ZHANG Long, LI Wenting, WONG Sai-Wai, DENG Wei, CHI Baoyong. Leaky-Wave Antennas for 5G/B5G Mobile Communication Systems: A Survey [J]. ZTE Communications, 2020, 18(3): 3-11.
[8]	XIA Chenhui, WANG Gang, WANG Bo, MING Xuefei. Integrated 3D Fan-out Package of RF Microsystem and Antenna for 5G Communications [J]. ZTE Communications, 2020, 18(3): 33-41.
[9]	LIU Xin, CHEN Wenhua, WANG Dehan, NING Dongfang. Robust Digital Predistortion for LTE/5G Power Amplifiers Utilizing Negative Feedback Iteration [J]. ZTE Communications, 2020, 18(3): 49-56.
[10]	GONG Shuhong, ZHANG Xingmin, DOU Jianwu, HUANG Weifang. Non‑Negligible Influences of Rain on 5G Millimeter Wave Terrestrial Communication System [J]. ZTE Communications, 2020, 18(3): 64-70.
[11]	WU Hequan. Ten Reflections on 5G [J]. ZTE Communications, 2020, 18(1): 1-4.
[12]	LIU Jianwei, YUAN Yifei, HAN Jing. A Case Study on Intelligent Operation System for Wireless Networks [J]. ZTE Communications, 2019, 17(4): 19-26.
[13]	HAN Bin, Hans D. SCHOTTEN. Machine Learning for Network Slicing Resource Management:A Comprehensive Survey [J]. ZTE Communications, 2019, 17(4): 27-32.
[14]	HUANG He, LIU Yang, LIU Zhuang, HAN Jiren, GAO Yin. Mechanism of Fast Data Retransmission in CU-DU Split Architecture of 5G NR [J]. ZTE Communications, 2018, 16(3): 40-44.
[15]	Zekeriyya Esat Ankaralı, Berker Peköz, Hüseyin Arslan. Enhanced OFDM for 5G RAN [J]. ZTE Communications, 2017, 15(S1): 11-20.

Indoor Environment and Human Sensing via Millimeter Wave Radio: A Review

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 7

References 23

Related Articles 15

Recommended Articles

Metrics

Comments