A study on deep learning for natural language generation in spoken dialogue systems
➤ Gửi thông báo lỗi ⚠️ Báo cáo tài liệu vi phạmNội dung chi tiết: A study on deep learning for natural language generation in spoken dialogue systems
A study on deep learning for natural language generation in spoken dialogue systems
Doctoral DissertationA Study on Deep Learning for Natural Language Generation in Spoken Dialogue SystemsTRAN Van KhanhSupervisor: Associate Professor A study on deep learning for natural language generation in spoken dialogue systems NGUYEN Le MinhSchool of Information ScienceJapan Advanced Institute of Science and TechnologySeptember. 2018To my wife, my daughter, and my family.Without whom I would never have completed this dissertation.AbstractNatural language generation (NĨ.G) plays a critical role in spoken dialogue systems A study on deep learning for natural language generation in spoken dialogue systems (SDSs) and aims al convening a meaning representation, i.e.. a dialogue act (DA), into natural language utterances. NĨ.G process in SDSs can typicallyA study on deep learning for natural language generation in spoken dialogue systems
be split up into two stages: sentence planning and surface realization. Sentence planning decides the order and structure of sentence representation.Doctoral DissertationA Study on Deep Learning for Natural Language Generation in Spoken Dialogue SystemsTRAN Van KhanhSupervisor: Associate Professor A study on deep learning for natural language generation in spoken dialogue systems sive hand-crafted rules and templates that arc time-consuming, expensive and do not generalize well. The resulting NLG systems, thus, tend to generate stiff responses, lacking several factors: adequacy, fluency and naturalness. Recent advances in data-driven and deep neural networks (DNNs) methods h A study on deep learning for natural language generation in spoken dialogue systems ave facilitated investigation of NLG in the study. DNN methods to NLG for SDS have demonstrated to generate belter responses than conventional methodsA study on deep learning for natural language generation in spoken dialogue systems
concerning factors as mentioned above. Nevertheless, when dealing with the NLG problems, such DNN-bascd NLG models still suffer from some severe drawDoctoral DissertationA Study on Deep Learning for Natural Language Generation in Spoken Dialogue SystemsTRAN Van KhanhSupervisor: Associate Professor A study on deep learning for natural language generation in spoken dialogue systems to tackle the problems of the existing DNN-based NLG models.Firstly, we present gating generators based on a recurrent neural network language model (RNNLM) to overcome the NLG problems of completeness. The proposed gales are intuitively similar to those in the Long short-term memory (LSTM) or Gate A study on deep learning for natural language generation in spoken dialogue systems d recurrent unit (GRU) to restrain the gradient vanishing and exploding. In our models, the proposed gates are in charge of sentence planning to decidA study on deep learning for natural language generation in spoken dialogue systems
e “How lo say it?”, whereas the RNNLM forms a surface realization to generate surface texts. More specifically, we introduce three additional semanticDoctoral DissertationA Study on Deep Learning for Natural Language Generation in Spoken Dialogue SystemsTRAN Van KhanhSupervisor: Associate Professor A study on deep learning for natural language generation in spoken dialogue systems an adjustment cell and an output cell arc to select semantic elements and to gale a feature vector DA during generation, respectively. '1 he proposed models further obtain state-of-the-art results over previous models regarding BLEU and slot error rale LRR scores.Secondly, we propose a novel hybrid A study on deep learning for natural language generation in spoken dialogue systems NI.G framework to address the first two NLG problems. which is an extension of an RNN Encoder-Decoder incorporating with an attention mechanism. TheA study on deep learning for natural language generation in spoken dialogue systems
idea of attention mechanism is to automatically learn alignments between features from source and target sentence during decoding. Our hybrid frameworDoctoral DissertationA Study on Deep Learning for Natural Language Generation in Spoken Dialogue SystemsTRAN Van KhanhSupervisor: Associate Professor A study on deep learning for natural language generation in spoken dialogue systems hanisms. Tn the first model. we introduce an additional cell into aligner cell by utilizing another attention or gating mechanisms to align and control the semantic elements produced by the encoder w ith a conventional attention mechanism over the inpul elements. In the second model, we develop a re A study on deep learning for natural language generation in spoken dialogue systems finement adjustment LSTM (RALSTM) decoder to select, aggregate semantic elements and to form the required utterances. The hybrid generators not only tA study on deep learning for natural language generation in spoken dialogue systems
ackle the NLG problems of completeness, achieving state-of-the-art performances over previous methods, but also deal with adaptability issue by showinDoctoral DissertationA Study on Deep Learning for Natural Language Generation in Spoken Dialogue SystemsTRAN Van KhanhSupervisor: Associate Professor A study on deep learning for natural language generation in spoken dialogue systems he problem of low-resource setting data in a domain adaptation scenario. The proposed models demonstrate an ability to perform acceptably well in a new. unseen domain by using only 10% amount of the target domain data. More precisely, we first present a variational generator by integrating a variati A study on deep learning for natural language generation in spoken dialogue systems onal autoencoder into the hybrid generator. We then propose two critics, namely domain, and text similarity, in an adversarial training algorithm to tA study on deep learning for natural language generation in spoken dialogue systems
rain the variational generator via multiple adaptation steps. The ablation experiments demonstrated that while the variational generator contributes tDoctoral DissertationA Study on Deep Learning for Natural Language Generation in Spoken Dialogue SystemsTRAN Van KhanhSupervisor: Associate Professor A study on deep learning for natural language generation in spoken dialogue systems the adversarial training procedure.Fourthly, we propose another approach dealing with the problem of having low-resource in-domain training data. The proposed generators, which combines two variational autoen-codcrs. can learn more efficiently when the training data is in short supply. In particula A study on deep learning for natural language generation in spoken dialogue systems rly, we present a combination of a variational generator with a variational CNN-DCNN. resulting in a generator which can perform acceptably well usingA study on deep learning for natural language generation in spoken dialogue systems
only 1()%I to 30% amount of in-domain training data. More importantly, the proposed model demonstrates state-of-the-art performance regarding BLEU anDoctoral DissertationA Study on Deep Learning for Natural Language Generation in Spoken Dialogue SystemsTRAN Van KhanhSupervisor: Associate Professor A study on deep learning for natural language generation in spoken dialogue systems ve contribution to learning the global semantic information of pairs of DA-utterance. the variational CNN-DCNN play a critical role of encoding useful information into the latent variable.Finally, all the proposed generators in this study can learn from unaligncd data by jointly training both senten A study on deep learning for natural language generation in spoken dialogue systems ce planning and surface realization to generate natural language utterances. Experiments further demonstrate that the proposed models achieved signifiA study on deep learning for natural language generation in spoken dialogue systems
cant improvements over previous generators concerning two evaluation metrics across four primary NLG domains and variants in a variety of training sceDoctoral DissertationA Study on Deep Learning for Natural Language Generation in Spoken Dialogue SystemsTRAN Van KhanhSupervisor: Associate Professor A study on deep learning for natural language generation in spoken dialogue systems tudy in the future.Keywords: natural language generation, spoken dialogue system, domain adaptation, gating mechanism, attention mechanism, encoder-decoder, low -resource data. RNN. GRU. LSTM. CNN. Dcconvolulional CNN. VAE.iiiAcknowledgementsI would like to thank my supervisor. Associate Professor N A study on deep learning for natural language generation in spoken dialogue systems guyen Ĩ.C Minh, for his guidance and motivation. He gave inc a lol of valuable and critical comments. advice and discussion, which foster me pursuingA study on deep learning for natural language generation in spoken dialogue systems
this research topic from the starting point. He always encourages and challenges me lo submit our works to the lop natural language processing confereDoctoral DissertationA Study on Deep Learning for Natural Language Generation in Spoken Dialogue SystemsTRAN Van KhanhSupervisor: Associate Professor A study on deep learning for natural language generation in spoken dialogue systems ver finished this research.I would also like to thank the tutors in writing lab at JATST: Terrillon Jean-Christophe, Bill Holden. Natt Ambassah and John Blake, who gave many useful comments on my manuscripts. 1 greatly appreciate useful comments from committee members: Professor Satoshi Tojo. Associ A study on deep learning for natural language generation in spoken dialogue systems ate Professor Kiyoaki Shirai. Associate Professor Shogo Okada, and Associate Professor Tran The Truycn.I must thank my colleagues in Nguyen's LaboratoA study on deep learning for natural language generation in spoken dialogue systems
ry for their valuable comments and discussion during the weekly seminar. I owe a debt of gratitude to all the members of the Vietnamese Football Club Doctoral DissertationA Study on Deep Learning for Natural Language Generation in Spoken Dialogue SystemsTRAN Van KhanhSupervisor: Associate Professor A study on deep learning for natural language generation in spoken dialogue systems g my favorite sports every week, which help me keep my physical health and recover my energy for pursuing research topic and surviving on the Ph.D. life.I appreciate anonymous reviewers from the conferences who gave me valuable and useful comments on my submilled papers, from which 1 could revise an A study on deep learning for natural language generation in spoken dialogue systems d improve my works. 1 am grateful for the funding source that allowed me to pursue this research: The Vietnamese Government’s Scholarship under the 91A study on deep learning for natural language generation in spoken dialogue systems
1 Project 'Training lecturers of Doctor's Degree for universities and colleges for the 2010-2020 period".Finally. I am deeply thankful to my family foDoctoral DissertationA Study on Deep Learning for Natural Language Generation in Spoken Dialogue SystemsTRAN Van KhanhSupervisor: Associate Professor A study on deep learning for natural language generation in spoken dialogue systems Tran Van Minh, my Mom. Nguyen Thi Luu, my younger sister. Tran Thi Dicu I.inh. and my parents in law for their constant love and support. This last word of acknowledgment I have saved for my dear wife Du Thi Ha and my lovely daughter Tran Thi Minh Khue. w ho always be on my side and encourage me to A study on deep learning for natural language generation in spoken dialogue systems look forward lo a better future.ivTable of ContentsAbstractiAcknowledgementsiTable of Contents3List of Figures4List of Tables5Gọi ngay
Chat zalo
Facebook