Deep Neural Network-Based Chinese Semantic Role Labeling

doi:10.3969/j.issn.1673-5188.2017.S2.010

ZTE Communications ›› 2017, Vol. 15 ›› Issue (S2): 58-64.DOI: 10.3969/j.issn.1673-5188.2017.S2.010

• Research Paper • Previous Articles

Deep Neural Network-Based Chinese Semantic Role Labeling

ZHENG Xiaoqing¹, CHEN Jun², SHANG Guoqiang²

1. School of Computer Science, Fudan University, Shanghai 201203, China
2. Terminal Business Division, ZTE Corporation, Shanghai 201203, China

Received:2016-11-24 Online:2017-12-25 Published:2020-04-16
About author:ZHENG Xiaoqing (zhengxq@fudan.edu.cn) received the Ph.D. degree in computer science from Zhejiang University, China. After then, he joined the faculty of School of Computer Science at Fudan University, China. He did research on semantic technology during his stay at the Information Technology Group, Massachusetts Institute of Technology (MIT), USA as an international faculty fellow from 2010 to 2011. His current research interests include natural language processing, deep learning, data integration and semantic web.|CHEN Jun (chen.jun_sh@zte.com.cn) received the Ph.D. Degree in signal and information processing from Xidian University, China. Since 2003, he joined ZTE corporation. His current research interests include image processing and deep learning technologies.|SHANG Guoqiang (shangguoqiang@zte.com.cn) received the B.S. Degree from Zhejiang University, China. Now, in ZTE his research interests include natural language processing, picture processing and so on.
Supported by:
This work was supported in part by a grant from ZTE Research Funding, and in part by a grant from Shanghai Municipal Natural Science Foundation(13ZR1403800);This work was supported in part by a grant from ZTE Research Funding, and in part by a grant from Shanghai Municipal Natural Science Foundation(15511104303)

Abstract

Abstract:

A recent trend in machine learning is to use deep architectures to discover multiple levels of features from data, which has achieved impressive results on various natural language processing (NLP) tasks. We propose a deep neural network-based solution to Chinese semantic role labeling (SRL) with its application on message analysis. The solution adopts a six-step strategy: text normalization, named entity recognition (NER), Chinese word segmentation and part-of-speech (POS) tagging, theme classification, SRL, and slot filling. For each step, a novel deep neural network-based model is designed and optimized, particularly for smart phone applications. Experiment results on all the NLP sub-tasks of the solution show that the proposed neural networks achieve state-of-the-art performance with the minimal computational cost. The speed advantage of deep neural networks makes them more competitive for large-scale applications or applications requiring real-time response, highlighting the potential of the proposed solution for practical NLP systems.

Key words: deep learning, sequence labeling, natural language understanding, convolutional neural network, recurrent neural network

ZHENG Xiaoqing, CHEN Jun, SHANG Guoqiang. Deep Neural Network-Based Chinese Semantic Role Labeling[J]. ZTE Communications, 2017, 15(S2): 58-64.

Figures/Tables 8

References 25

[1]	N. W. Xue , “Labeling Chinese predicates with semantic roles,” Computational Linguistics, vol. 34, no. 2, pp. 225-255, 2008. doi: 10.1162/coli.2008.34.2.225.
[2]	M. Surdeanu, S. Harabagiu, J. Williams, P. Aarseth , “Using predicate-argument structures for information extraction,” in Proc. The Annual Meeting of the Association for Computational Linguistics, Sapporo, Japan, Jul. 2003, pp. 8-15. doi: 10.3115/1075096.1075098.
[3]	G. Melli, Y. Wang, Y. Liu , et al., “Descrition of SQUASH, the SFU question and summery handler for the DUC-2005 summarization task,” in Document Understanding Conference, 2005.
[4]	H. C. Boas , “Bilingual framenet dictionaries for machine translation,” in Proc. International Conference on Language Resources and Evaluation, Jan. 2002, pp. 1364-1371.
[5]	S. Narayanan, S. Harabagiu , “Question answering based on semantic structures,” in Proc. 20th International Conference on Computational Linguistics, Geneva, Switzerland, Aug. 2004. doi: 10.3115/1220355.1220455.
[6]	S. Pradhan, W. Ward, K. Hacioglu, J. H. Martin , “Semantic role labeling using different syntactic views,” in Proc. International Conference on Computational Linguistics, USA, Michigan, Jun. 2005, pp. 581-588. doi: 10.3115/1219840.1219912.
[7]	J. L. Packard , The Morphology of Chinese: a Linguistic and Cognitive Approach, Cambridge, United Kingdom: Cambridge University Press, 2004.
[8]	N. Xue and M. Palmer , “Annotating the propositions in the Penn Chinese Treebank,” in Proc. The 2nd SIGHAN Workshop on Chinese Language Processing, Sapporo, Japan, Jul. 2003, pp. 47-54. doi: 10.3115/1119250.1119257.
[9]	Y. Bengio , “Learning deep architectures for AI,” Foundations and Trends in Machine Learning, vol. 2, no. 1, pp. 1-127, 2009. doi: 10.1561/2200000006.
[10]	G. F. Luger , Artificial Intelligence: Structures and Strategies for Complex Problem Solving 5th edition. New Jersey, USA: Addison Wesley, 2004.
[11]	H. T. Ng, J. K. Lou , “Chinese part-of-speech tagging: one-at-a-time or all-at-once? word-based or character-based?” in Proc. Conference on Empirical Methods in Natural Language Processing, Jul. 2004, pp. 277-284.
[12]	R. Collobert, J. Weston, L. Bottou , et al., “Natural language processing (almost) from scratch,” Journal of Machine Learning Research, vol. 12, no. 1, pp. 2493-2537, 2011.
[13]	X. Zheng, H. Chen, T. Xu , “Deep learning for Chinese word segmentation and POS tagging,” in Proc. Conference on Empirical Methods in Natural Language Processing, Oct. 2013, pp. 647-657.
[14]	N. Kalchbrenner, E. Grefenstette, P. Blunsom , “A convolutional neural network for modelling sentences,” in Proc. Annual Meeting of the Association for Computational Linguistics, Apr. 2014, pp. 655-665.
[15]	S. Hochreiter, J. Schmidhuber , “Long short-term memory,” Neural Computation, vol. 9, no. 8, pp. 1735-1780, 1997. doi: 10.1162/neco.1997.9.8.1735.
[16]	Z. Huang, W. Xu, K. Yu. (2015, Aug.). Bidirectional LSTM-CRF models for sequence tagging [Online]. Available: http://arxiv.org/abs/1508.01991v1
[17]	A. Graves, A. Mohamed, G. Hinton. (2013, Mar.). Speech recognition with deep recurrent neural networks [Online]. Available: http://arxiv.org/abs/1303.5778
[18]	M. Collins , “Discriminative training methods for hidden Markov models: theory and experiments with perceptron algorithms,” in Proc. Conference on Empirical Methods in Natural Language Processing, 2002, pp. 1-8. doi: 10.3115/1118693.1118694.
[19]	R. Socher, C. C-Y. Lin, A. Y. Ng, and D. Christopher , “Parsing natural scenes and natural language with recursive neural networks,” in Proc. International Conference on Machine Learning, Washington, USA, Jun. 2011, pp. 129-136.
[20]	W. Pei, T. Ge, B. Chang , “Max-margin tensor neural network for Chinese word segmentation,” in Proc. The Annual Meeting of the Association for Computational Linguistics, 2014, doi: 10.3115/v1/P14-1028.
[21]	D. Erhan, Y. Bengio, A. Courville, P. Manzagol , et al., “Why does unsupervised pre-training help deep learning,” Journal of Machine Learning Research, vol.11, pp. 625-660, 2010.
[22]	T. Mikolov, K. Chen, G. Corrado, J. Dean. (2013, Jan.). “Efficient estimation of word representations in vector spaces [Online]. Available: http://arxiv.org/abs/1301.3781
[23]	G. A. Levow , “The third international Chinese language processing bakeoff: word segmentation and named entity recognition,” in Proc. SIGHAN Workshop on Chinese Language Processing, 2006, pp. 108-117.
[24]	R. T. Tsai, H. C. Hung, C. Sung, H. Dai , et al., “On closed task of Chinese word segmentation: an improved CRF model coupled with character clustering and automatically generated template matching,” in Proc. The Fifth SIGHAN Workshop on Chinese Language Processing, 2006, pp. 108-117.
[25]	H. Zhao, C. N. Huang, M. Li , “An improved Chinese word segmentation system with conditional random field,” in Proc. The Fifth SIGHAN Workshop on Chinese Language Processing, 2006, pp. 162-165.

Words	Labels
明天 “tomorrow”	DATE
上午九点 “9 am”	TIME
在 “at”	O
第一会议室 “meeting room No. 1”	LOCATION
我们 “we”	O
与 “with”	O
技术部 “technology section”	PARTICIPANT
开会 “have a meeting”	O
讨论 “discuss”	O
项目进展 “the progress of the project”	TOPIC
。 “.”	O

Words	Labels
明天 “tomorrow”	DATE
上午九点 “9 am”	TIME
在 “at”	O
第一会议室 “meeting room No. 1”	LOCATION
我们 “we”	O
与 “with”	O
技术部 “technology section”	PARTICIPANT
开会 “have a meeting”	O
讨论 “discuss”	O
项目进展 “the progress of the project”	TOPIC
。 “.”	O

Task	Goal	Model
Word segmentation (F1)	~ 85	≥ 90
POS tagging (F1)	~ 80	≥ 88
Named entity recognition (F1)	~ 75	≥ 84
SRL (F1)	~ 70	≥ 80

Task	Goal	Model
Word segmentation (F1)	~ 85	≥ 90
POS tagging (F1)	~ 80	≥ 88
Named entity recognition (F1)	~ 75	≥ 84
SRL (F1)	~ 70	≥ 80

System	Parameters	Time(ms)
Tsai et al. [24]	3027k	602
Zhao et al. [25]	3711k	859
Neural network	459k	49

Deep Neural Network-Based Chinese Semantic Role Labeling

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 8

References 25

Related Articles 13

Recommended Articles

Metrics

[1]	WANG Chongchong, LI Yao, WANG Beibei, CAO Hong, ZHANG Yanyong. Point Cloud Processing Methods for 3D Point Cloud Detection Tasks [J]. ZTE Communications, 2023, 21(4): 38-46.
[2]	GONG Panyin, ZHANG Guidong, ZHANG Zhigang, CHEN Xiao, DING Xuan. Research on Fall Detection System Based on Commercial Wi-Fi Devices [J]. ZTE Communications, 2023, 21(4): 60-68.
[3]	DENG Letian, ZHAO Yanru. Deep Learning-Based Semantic Feature Extraction: A Literature Review and Future Directions [J]. ZTE Communications, 2023, 21(2): 11-17.
[4]	LU Ping, SHENG Bin, SHI Wenzhe. Scene Visual Perception and AR Navigation Applications [J]. ZTE Communications, 2023, 21(1): 81-88.
[5]	FAN Guotian, WANG Zhibin. Intelligent Antenna Attitude Parameters Measurement Based on Deep Learning SSD Model [J]. ZTE Communications, 2022, 20(S1): 36-43.
[6]	GAO Zhengguang, LI Lun, WU Hao, TU Xuezhen, HAN Bingtao. A Unified Deep Learning Method for CSI Feedback in Massive MIMO Systems [J]. ZTE Communications, 2022, 20(4): 110-115.
[7]	ZHANG Jintao, HE Zhenqing, RUI Hua, XU Xiaojing. Spectrum Sensing for OFDMA Using Multicarrier Covariance Matrix Aware CNN [J]. ZTE Communications, 2022, 20(3): 61-69.
[8]	HE Hongye, YANG Zhiguo, CHEN Xiangning. Payload Encoding Representation from Transformer for Encrypted Traffic Classification [J]. ZTE Communications, 2021, 19(4): 90-97.
[9]	ZHANG Chenchen, ZHANG Nan, CAO Wei, TIAN Kaibo, YANG Zhen. AI-Based Optimization of Handover Strategy in Non-Terrestrial Networks [J]. ZTE Communications, 2021, 19(4): 98-104.
[10]	Julian AHRENS, Lia AHRENS, Hans D. SCHOTTEN. A Machine Learning Method for Prediction of Multipath Channels [J]. ZTE Communications, 2019, 17(4): 12-18.
[11]	XUE Songyan, LI Ang, WANG Jinfei, YI Na, MA Yi, Rahim TAFAZOLLI, Terence DODGSON. To Learn or Not to Learn:Deep Learning Assisted Wireless Modem Design [J]. ZTE Communications, 2019, 17(4): 3-11.
[12]	WANG Shihao, ZHUO Qinzheng, YAN Han, LI Qianmu, QI Yong. A Network Traffic Prediction Method Based on LSTM [J]. ZTE Communications, 2019, 17(2): 19-25.
[13]	ZHENG Xiaoqing, LU Yaping, PENG Haoyuan, FENG Jiangtao, ZHOU Yi, JIANG Min, MA Li, ZHANG Ji, JI Jie. Detecting Abnormal Start-Ups, Unusual Resource Consumptions of the Smart Phone: A Deep Learning Approach [J]. ZTE Communications, 2019, 17(2): 38-43.

System	Parameters	Time(ms)
CRFs-based system	28k	~600
LSTM (with 20 cells)	6k	~500

System	Parameters	Time(ms)
CRFs-based system	28k	~600
LSTM (with 20 cells)	6k	~500