ZTE Communications ›› 2022, Vol. 20 ›› Issue (S1): 27-35.doi: 10.12142/ZTECOM.2022S1005

• Research Paper • Previous Articles     Next Articles

End-to-End Chinese Entity Recognition Based on BERT-BiLSTM-ATT-CRF

LI Daiyi1(), TU Yaofeng2, ZHOU Xiangsheng2, ZHANG Yangming2, MA Zongmin1   

  1. 1.Nanjing University of Aeronautics and Astronautics, Nanjing 211106, China
    2.ZTE Corporation, Shenzhen 518057, China
  • Online:2022-01-25 Published:2022-03-01
  • About author:LI Daiyi (lidaiyi@nuaa.edu.cn) received his master’s degree from School of computer and Communication Engineering, Zhengzhou University of Light Industry, China in 2018. He is studying for a doctor’s degree in the School of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, China. His main research interests are knowledge graphs and big data.|TU Yaofeng received his Ph.D. degree from Nanjing University of Aeronautics and Astronautics, China. He is a researcher at ZTE Corporation. His research interests include big data, database and machine learning.|ZHOU Xiangsheng is an expert and senior R&D manager in the AI field of ZTE Corporation. His research fields mainly include NLP, NAS, training acceleration, etc.|ZHANG Yangming is a software engineer at ZTE Corporation. His research interests mainly focus on natural language processing, knowledge engineering and acoustic signal processing.|MA Zongmin received his Ph.D. degree from the City University of Hong Kong, China and is a full professor with Nanjing University of Aeronautics and Astronautics, China. His research interests mainly include big data and knowledge engineering. He has published more than 100 papers in highly cited international journals and authored five monographs published by Springer. He is the Fellow of IFSA and Fellow of IET.
  • Supported by:
    ZTE Industry-University-Institute Cooperation Funds(HC-CN-20190910009);the National Natural Science Foundation of China(61772269)


Traditional named entity recognition methods need professional domain knowledge and a large amount of human participation to extract features, as well as the Chinese named entity recognition method based on a neural network model, which brings the problem that vector representation is too singular in the process of character vector representation. To solve the above problem, we propose a Chinese named entity recognition method based on the BERT-BiLSTM-ATT-CRF model. Firstly, we use the bidirectional encoder representations from transformers (BERT) pre-training language model to obtain the semantic vector of the word according to the context information of the word; Secondly, the word vectors trained by BERT are input into the bidirectional long-term and short-term memory network embedded with attention mechanism (BiLSTM-ATT) to capture the most important semantic information in the sentence; Finally, the conditional random field (CRF) is used to learn the dependence between adjacent tags to obtain the global optimal sentence level tag sequence. The experimental results show that the proposed model achieves state-of-the-art performance on both Microsoft Research Asia (MSRA) corpus and people’s daily corpus, with F1 values of 94.77% and 95.97% respectively.

Key words: named entity recognition (NER), feature extraction, BERT model, BiLSTM, attention mechanism, CRF