基于LSTM-Attention神经网络的文本特征提取方法Text feature extraction method based on LSTM-Attention neural network
赵勤鲁,蔡晓东,李波,吕璐
摘要(Abstract)：
针对当前文本分类神经网络不能充分提取词语与词语和句子与句子之间的语义结构特征信息的问题,提出一种基于LSTM-Attention的神经网络实现文本特征提取的方法。首先,分别使用LSTM网络对文本的词语与词语和句子与句子的特征信息进行提取;其次,使用分层的注意力机制网络层分别对文本中重要的词语和句子进行选择;最后,将网络逐层提取得到的文本特征向量使用softmax分类器进行文本分类。实验结果表明,所提方法可以有效地提取文本的特征,使得准确率得到提高。将该方法应用在IMDB,yelp2013和yelp2014数据集上进行实验,分别得到52.4%,66.0%和67.6%的正确率。
关键词(KeyWords)： LSTM-Attention;注意力机制;文本分类;神经网络;文本特征提取;softmax
基金项目(Foundation): 广西科技计划项目(广西重点研发计划)(桂科AB16380264);; 国家科技支撑计划课题(2014BAK11B02)~~
作者(Author): 赵勤鲁,蔡晓东,李波,吕璐
DOI: 10.16652/j.issn.1004-373x.2018.08.041
参考文献(References)：
- [1]WAWRE S V,DESHMUKH S N.Sentiment classification using machine learning techniques[J].International journal of science and research,2016,5(4):819-821.
- [2]DING X,LIU B,YU P S.A holistic lexicon-based approach to opinion mining[C]//Proceedings of the 2008 International Conference on Web Search and Data Mining.New York:Association for Computing Machinery,2008:231-240.
- [3]KIM Y.Convolutional neural networks for sentence classification[J/OL].[2014-09-03].https://arxiv.org/pdf/1408.5882v2.pdf.
- [4]ZHANG X,ZHAO J,LECUN Y.Character-level convolutional networks for text classification[J/OL].[2017-07-09].http://www.doc88.com/p-1582873490700.html.
- [5]梁军,柴玉梅,原慧斌,等.基于极性转移和LSTM递归网络的情感分析[J].中文信息学报,2015,29(5):152-159.LIANG Jun,CHAI Yumei,YUAN Huibin,et al.Polarity shifting and LSTM based recursive networks for sentiment analysis[J].Journal of Chinese information processing,2015,29(5):152-159.
- [6]TANG D,QIN B,LIU T.Document modeling with gated recurrent neural network for sentiment classification[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.[S.l.:s.n.],2015:1422-1432.
- [7]BAHDANAU D,CHO K,BENGIO Y.Neural machine translation by jointly learning to align and translate[J/OL].[2014-10-11].http://nlp.ict.ac.cn/Admin/kindeditor/attached/file/20141011/20141011133445_31922.pdf.
- [8]DOS-SANTOS C,TAN M,XIANG B,et al.Attentive pooling networks[J/OL].[2016-02-11].http://xueshu.baidu.com/s?wd=paperuri%3A%28f618ebd90de235437fd745ed36080e77%&29filter=sc_long_sign&tn=SE_xueshusource_2kduw22v&sc_vurl=http%3A%2F%2Farxiv.org%2Fabs%2F1602.03609&ie=utf-8&sc_us=17197570488667881682.
- [9]HOCHREITER S,SCHMIDHUBER J.Long short-term memory[J].Neural computation,1997,9(8):1735-1780.
- [10]MIKOLOV T,SUTSKEVER I,CHEN K,et al.Distributed representations of words and phrases and their compositionality[J/OL].[2013-10-16].https://arxiv.org/pdf/1310.4546v1.pdf.