循环神经网络中基于特征融合的口语理解Feature fusion based recurrent neural network for spoken language understanding
张晶晶,黄浩,胡英,吾守尔·斯拉木
摘要(Abstract)：
口语理解(SLU)性能的好坏对口语对话系统有着至关重要的作用。在对基本循环神经网络及其变体长短时记忆(LSTM)网络和门限循环单元(GRU)网络结构分析的基础上,提出一种特征融合的循环神经网络结构。该结构首先把输入送到隐含层训练得到特征表示;然后该特征信息与源输入及历史输出信息一起送入另一隐含层训练;最后送进输出层得出结果。对上述不同的循环神经网络的结构及提出的模型在ATIS数据库上进行口语理解实验。结果表明,提出的特征融合的循环神经网络结构的性能要优于传统循环神经网络及其变体结构。
关键词(KeyWords)： 口语理解;循环神经网络;长短时记忆;门限循环单元;特征融合;自然语言
基金项目(Foundation): 国家自然科学基金(61365005);国家自然科学基金(61663044);国家自然科学基金(61761041);; 新疆大学博士科研启动基金(BS160239)~~
作者(Author): 张晶晶,黄浩,胡英,吾守尔·斯拉木
DOI: 10.16652/j.issn.1004-373x.2018.20.038
参考文献(References)：
- [1]俞凯,陈露,陈博,等.任务型人机对话系统中的认知技术:概念、进展及其未来[J].计算机学报,2015,38(12):2333-2348.YU Kai,CHEN Lu,CHEN Bo,et al. Cognitive technology in task-oriented dialogue systems:concepts,advances and future[J]. Chinese journal of computers,2015,38(12):2333-2348.
- [2] HOCHREITER S,SCHMIDHUBER J. Long short-term memory[J]. Neural computation,1997,9(8):1735-1780.
- [3] CHO K, VAN MERRI?NBOER B, GULCEHRE C, et al.Learning phrase representations using RNN encoder-decoder for statistical machine translation[J/OL].[2014-09-03]. http://xueshu.baidu.com/s?wd=paperuri%3A%282ba31a0b87d74d25317da3521cca2915%29&filter=sc_long_sign&tn=SE_xueshusource_2kduw22v&sc_vurl=http%3A%2F%2Farxiv. org%2Fabs%2F1406.1078&ie=utf-8&sc_us=9851577233160611135.
- [4] MESNIL G,HE X,DENG L,et al. Investigation of recurrentneural-network architectures and learning methods for spoken language understanding[J/OL].[2013-08-01]. https://www.microsoft. com/en-us/research/wp-content/uploads/2016/02/IS13_RNNSLU_cameraready.pdf.
- [5] MESNIL G,DAUPHIN Y,YAO K,et al. Using recurrent neural networks for slot filling in spoken language understanding[J]. IEEE/ACM transactions on audio,speech,and language processing,2015,23(3):530-539.
- [6] YAO K,ZWEIG G,HWANG M Y,et al. Recurrent neural networks for language understanding[C/OL].[2014-10-14].https://www. researchgate. net/publication/266798083_Recurrent_Neural_Networks_for_Language_Understanding.
- [7] JOZEFOWICZ R,ZAREMBA W,SUTSKEVER I. An empirical exploration of recurrent network architectures[C]//Proceedings of 32nd International Conference on Machine Learning.Lille:Association for Computing Machinery,2015:2342-2350.
- [8] AL-RFOU R,ALAIN G,ALMAHAIRI A,et al. Theano:a Python framework for fast computation of mathematical expressions[J/OL].[2016-05-09]. https://www.researchgate.net/publication/302569301/download.
- [9] SUTTON C,MCCALLUM A. An introduction to conditional random fields[J]. Foundations and trends in machine learning,2011,4(4):267-373.
- [10] MIKOLOV T,JOULIN A,CHOPRA S,et al. Learning longer memory in recurrent neural networks[J/OL].[2015-04-16].https://research. fb. com/wp-content/uploads/2016/11/learning_longer_memory_in_recurrent_neural_networks.pdf?.
- [11] HENDERSON M,GA?I?M,THOMSON B,et al. Discriminative spoken language understanding using word confusion networks[C]//Proceedings of IEEE Spoken Language Technology Workshop. Miami:IEEE,2013:176-181.