HiddenTag: Enabling Person Identification Without Privacy Exposure

doi:10.12142/ZTECOM.202103002

ZTE Communications ›› 2021, Vol. 19 ›› Issue (3): 3-12.DOI: 10.12142/ZTECOM.202103002

• Special Topic • Previous Articles Next Articles

HiddenTag: Enabling Person Identification Without Privacy Exposure

QIU Chen¹(), DAI Tao², GUO Bin¹, YU Zhiwen¹, LIU Sicong¹

^1.Northwestern Polytechnical University, Xi’an 710072, China
^2.Chang’an University, Xi’an 710064, China

Received:2021-06-15 Online:2021-09-25 Published:2021-10-11
About author:QIU Chen (qiuchen@nwpu.edu.cn) received the Ph.D. degree in computer science from Michigan State University, USA in 2017. He is currently an associate professor with Northwestern Polytechnical University, China. His research interests include pervasive computing, mobile computing, and applied machine learning. He is a member of the IEEE.|DAI Tao received his B.S., M.S. and Ph.D. degrees in software engineering from Xi’an Jiaotong University, China in 2008, 2011 and 2020, respectively. He is currently a lecturer at the School of Economics and Management， Chang’an University, China. He was a visiting student at the School of Computer Science, Carnegie Mellon University, USA from September 2018 to September 2019. His main research interests include natural language processing, information retrieval, and machine learning.|GUO Bin received the Ph.D. degree in computer science from Keio University, Japan in 2009 and then went to the French National Institute of Telecommunications for postdoctoral research. He is a professor with Northwestern Polytechnical University, China. His research interests include ubiquitous computing, mobile crowd sensing, and HCI. He is a senior member of the IEEE.|YU Zhiwen received the Ph.D. degree from Northwestern Polytechnical University, China. He is currently a professor and Dean with the School of Computer Science, Northwestern Polytechnical University. His research interests include pervasive computing and human-computer interaction. He is a senior member of the IEEE.|LIU Sicong received the B.S., M.S. and Ph.D. degrees from Xidian University, China in 2013, 2016, and 2020 respectively. From 2017 to 2018, she was a visiting scholar at Rice University, USA. She is currently an associate professor with Northwestern Polytechnical University, China. Her research interests include mobile computing system, mobile and embedded deep learning design, and automated deep model optimization.

Abstract

Abstract:

Person identification is the key to enable personalized services in smart homes, including the smart voice assistant, augmented reality, and targeted advertisement. Although research in the past decades in person identification has brought technologies with high accuracy, existing solutions either require explicit user interaction or rely on images and video processing, and thus suffer from cost and privacy limitations. In this paper, we introduce a device-free personal identification system–HiddenTag, which utilizes smartphones to identify different users via profiling indoor activities with inaudible sound and channel state information (CSI). HiddenTag sends inaudible sound and senses its diffraction and multi-path reflection using smartphones. Based upon the multi-path effects and human body absorption, we design suitable sound signals and acoustic features for constructing the human body signatures. In addition, we use CSI to trigger the system of acoustic sensing. Extensive experiments indicate that HiddenTag can distinguish multi-person in 10–15 s with 95.1% accuracy. We implement a prototype of HiddenTag with an online system by Android smartphones and maintain 84%–90% online accuracy.

Key words: person identification, acoustic sensing, CSI, smart home

QIU Chen, DAI Tao, GUO Bin, YU Zhiwen, LIU Sicong. HiddenTag: Enabling Person Identification Without Privacy Exposure[J]. ZTE Communications, 2021, 19(3): 3-12.

Figures/Tables 13

References 28

1	CLIFFORD B R, BULL R. The psychology of person identification [M]. London, UK: Routledge, 2017. DOI: 10.4324/9781315533537 DOI
2	BADRINARAYANANN V A, SIERRA J J, MARTIN K M. A dual identification framework of online multiplayer video games: The case of massively multiplayer online role playing games (MMORPGs) [J]. Journal of business research, 2015, 68(5): 1045-1052
3	FANG B Y, CO J, ZHANG M. DeepASL: enabling ubiquitous and non‑intrusive word and sentence‑level sign language translation [C]//The 15th ACM Conference on Embedded Network Sensor Systems. Delft, Netherlands: ACM, 2017: 1–13. DOI: 10.1145/3131672.3131693 DOI
4	ALI K, LIU A X, WEI W, et al. Keystroke Recognition Using WiFi Signals[C]// ACM MobiCom. Paris, France: ACM, 2015
5	YI J, LEE Y. Heimdall: mobile GPU coordination platform for augmented reality applications [C]//The 26th Annual International Conference on Mobile Computing and Networking. London, United Kingdom: ACM, 2020: 1–14. DOI: 10.1145/3372224.3419192 DOI
6	LSUN Y, LIANG D, WANG X G, et al. DeepID3: face recognition with very deep neural networks [J]. Computer Science, 2015
7	WANG M, DENG W H. Deep face recognition: a survey [EB/OL]. [2021-03-20].
8	ZHANG Z Y. Microsoft Kinect sensor and its effect [J]. IEEE multimedia, 19(2):4–10, 2012
9	LÓPEZ G, QUESADA L, GUERRERO L A. Alexa vs. Siri vs. Cortana vs. Google assistant: a comparison of speech-based natural user interfaces [C]//The AHFE 2019 International Conference on Human Factors and Systems Interaction. Washington, USA: AHFE, 2019. DOI: 10.1007/978-3-319-60366-7_23 DOI
10	HALPERIN D, HU W J, SHETH A, et al. Tool release [J]. ACM SIGCOMM computer communication review, 2011, 41(1): 53. DOI: 10.1145/1925861.1925870 DOI
11	WANG X Y, GAO L J, MAO S W. CSI phase fingerprinting for indoor localization with a deep learning approach [J]. IEEE Internet of Things journal, 2016, 3(6): 1113–1123. DOI: 10.1109/JIOT.2016.2558659 DOI
12	LOGAN B. Mel frequency cepstral coefficients for music modeling [EB/OL]. [2021-03-20].
13	SMCKINNEY M, BREEBAART J. Features for audio and music classification [EB/OL]. [2021-04-02].
14	TURK M, PENTLAND A. Eigenfaces for recognition [J]. Journal of cognitive neuroscience, 1991, 3(1): 71–86. DOI: 10.1162/jocn.1991.3.1.71 DOI
15	XIAO T, LI H S, OUYANG W L, et al. Learning deep feature representations with domain guided dropout for person re-identification [C]//IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE, 2016: 1249–1258. DOI: 10.1109/CVPR.2016.140 DOI
16	YU H X, ZHENG W S. Weakly supervised discriminative feature learning with state information for person identification [C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE, 2020: 5527–5537. DOI: 10.1109/CVPR42600.2020.00557 DOI
17	MUDA L, BEGAM M, ELAMVAZUTHI I. Voice recognition algorithms using mel frequency cepstral coefficient (MFCC) and dynamic time warping (DTW) techniques [EB/OL]. [2021-04-02].
18	BRUNELLI R, FALAVIGNA D. Person identification using multiple cues [J]. IEEE transactions on pattern analysis and machine intelligence, 1995, 17(10): 955–966. DOI: 10.1109/34.464560 DOI
19	PERALTA D, TRIGUERO I, SANCHEZ-REILLO R, et al. Fast fingerprint identification for large databases [J]. Pattern recognition, 47: 588–602, 2014. 10.1016/j.patcog.2013.08.002
20	RAO G S, NAGARAJU C, REDDY L, et al. A novel fingerprints identification system based on the edge detection [J]. International journal of computer science and network security, 8: 394– 397, 2008
21	TROJE N F, WESTHOFF C, LAVROV M. Person identification from biological motion: effects of structural and kinematic cues [J]. Perception & psychophysics, 2005, 67(4): 667–675. DOI: 10.3758/BF03193523 DOI
22	WANG Z, YU Z W, LOU X Y, et al. Gesture-radar: a dual Doppler radar based system for robust recognition and quantitative profiling of human gestures [J]. IEEE transactions on human-machine systems, 2021, 51(1): 32–43. DOI: 10.1109/THMS.2020.3036637 DOI
23	ZENG Y Z, PATHAK P H, MOHAPATRA P. WiWho: WiFi-based person identification in smart spaces [C]//15th ACM/IEEE International Conference on Information Processing in Sensor Networks. Vienna, Austria: IEEE, 2016: 1-12. DOI: 10.1109/IPSN.2016.7460727 DOI
24	MOON Y, KIM K J, SHIN D H. Voices of the Internet of Things: an exploration of multiple voice effects in smart homes [M]//Distributed, ambient and pervasive interactions. Cham: Springer International Publishing, 2016: 270–278. DOI: 10.1007/978-3-319-39862-4_25 DOI
25	TUNG Y C, SHIN K G. EchoTag: accurate infrastructure-free indoor location tagging with smartphones [C]//The 21st Annual International Conference on Mobile Computing and Networking. Paris, France: ACM, 2015: 525–536. DOI: 10.1145/2789168.2790102 DOI
26	YANG Z J, WEI Y L, SHEN S, et al. Ear-AR: Indoor acoustic augmented reality on earphones [C]//The 26th Annual International Conference on Mobile Computing and Networking. London, United Kingdom: ACM, 2020: 1–14. DOI: 10.1145/3372224.3419213 DOI
27	ZHOU B, ELBADRY M, GAO R P, et al. BatMapper: acoustic sensing based indoor floor plan construction using smartphones [C]//The 15th Annual International Conference on Mobile Systems, Applications, and Services. Niagara Falls, USA: ACM, 2017: 42–55. DOI: 10.1145/3081333.3081363 DOI
28	GEIGER J T, KNEIßL M, SCHULLER B W, et al. Acoustic gait-based person identification using hidden Markov models [C]//The 2014 Workshop on Mapping Personality Traits Challenge and Workshop. Istanbul, Turkey: ACM, 2014: 25–30. DOI: 10.1145/2668024.2668027 DOI

Features	Explanation
Crest factor	The value indicates how extreme the peaks are in a waveform
Energy	The energy of the signal in the time domain
Entropy of energy	The entropy of energy in the time domain
Spectral centroid	The center of the gravity of the frequency domain spectra
Spectral spread	The average spread of the spectrum in relation to its centroid
Spectral roll-off	The frequency below 90% of the magnitude distribution of the spectrum is concentrated
Spectral flux	The squared difference between two successive spectral frames
Spectral entropy	The entropy of the spectral energies
Spectral flatness	The ratio of the geometric mean to the arithmetic means of a power spectrum
Zero crossing rate	The rate of sign-changes along with a signal

Features	Explanation
Crest factor	The value indicates how extreme the peaks are in a waveform
Energy	The energy of the signal in the time domain
Entropy of energy	The entropy of energy in the time domain
Spectral centroid	The center of the gravity of the frequency domain spectra
Spectral spread	The average spread of the spectrum in relation to its centroid
Spectral roll-off	The frequency below 90% of the magnitude distribution of the spectrum is concentrated
Spectral flux	The squared difference between two successive spectral frames
Spectral entropy	The entropy of the spectral energies
Spectral flatness	The ratio of the geometric mean to the arithmetic means of a power spectrum
Zero crossing rate	The rate of sign-changes along with a signal

User	A	B	C	D
Height/cm	176	177	174	163
Weight/kg	65	80	70	55
Age	25	31	33	40
Gender	M	M	F	F

User	A	B	C	D
Height/cm	176	177	174	163
Weight/kg	65	80	70	55
Age	25	31	33	40
Gender	M	M	F	F

Actual/Classified	A	B	C	D
A	93.0%	1.0%	0.0%	6.0%
B	1.0%	98.0%	0.0%	1.0%
C	0.0%	0.0%	96.0%	4.0%
D	5.0%	16.0%	0.0%	79.0%

HiddenTag: Enabling Person Identification Without Privacy Exposure

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 13

References 28

Related Articles 3

Recommended Articles

Metrics

	Classified
Actual		1st	2nd	3rd	4th
	1st	48.0%	6.0%	3.0%	43.0%
	2nd	11.0%	74.0%	8.0%	7.0%
	3rd	1.0%	5.0%	74.0%	20.0%
	4th	40.0%	0.0%	7.0%	53.0%

	Information Type	Training Cost	Hardwares Required	Privacy Level	Accuracy
DeepID3	Image	High	Cameras	Low	92% offline
WiWho	CSI	Normal (100 s)	Special CSI devices	High	80%–92% offline
Step sound	Normal sound	Low (3 s)	Built-in smartphones	High	65% offline
HiddenTag	High frequency sound (18–21 kHz)	Normal (60 s)	Built-in smartphones	High	96% offline, 85%–90% online

[1]	TIAN Zengshan, YE Chenglin, ZHANG Gongzhui, HE Wei, JIN Yue. Speed Estimation Using Commercial Wi-Fi Device in Smart Home [J]. ZTE Communications, 2021, 19(2): 44-52.
[2]	CAO Jie, XU Lanyu, Raef Abdallah, SHI Weisong. An OS for Internet of Everything: Early Experience from A Smart Home Prototype [J]. ZTE Communications, 2017, 15(4): 12-22.
[3]	Andreas Brauchli, Depeng Li. A Solution-Based Analysis of Attack Vectors on Smart Home Systems [J]. ZTE Communications, 2015, 13(3): 6-12.

	Sweeping	Single-Tone	Multi-Tone
Offline	95.2%	91.0%	93.1%
Online	90.0%	80.0%	85.0%

	Sweeping	Single-Tone	Multi-Tone
Offline	95.2%	91.0%	93.1%
Online	90.0%	80.0%	85.0%