Loading...
| Friend's email | |
| Your name | |
| Your email | |
| enter code | |
This page was sent successfuly
- Type of Document: Ph.D. Dissertation
- Language: Farsi
- Document No: 55896 (19)
- University: Sharif University of Technology
- Department: Computer Engineering
- Advisor(s): Sameti, Hossein
- Abstract:
- Abstract: This thesis focuses on generating text from data. The Data-to-Text system must have three capabilities; First, it should be able to produce coherent, comprehensible, fluent text that is close to human natural language, in such a way that it is not possible to distinguish it from texts written by humans. Second, to be able to produce a variety of sentences to express the same concept. The third is to be able to express the information of the input data without repetition, redundancy, and omission in the output sentences. The latter is one of the main challenges of data-to-text systems because not being faithful to the input data can lead to se- rious problems in real-world applications. Until now, much research and studies have been done to fix or reduce the semantic defects in the output of the Data-to-Text generation mod- els. These studies include approaches such as using patterns and rules to generate sentences, using grammatical rules, using recurrent neural networks, and pre-trained language models. Although these approaches have been able to achieve some success, the problem of generat- ing sentences faithful to the input data has not been completely solved. For this purpose, in this thesis, we presented three new models that the sentences generated by them, while being coherent and fluent, have a high semantic quality. In the first model, for the given meaning representations, we create the tree structure of their equivalent sentences using dependency information and by applying strict restrictions on only adding input meaning labels to the tree. This tree structure only contains the input meaning labels, the part of speech tags, and the dependency relations between them. Then by replacing the nodes of the tree with suitable words and then traversing it, the output sentences are generated. This model, while generat- ing sentences without semantic defects, also improves the linguistic quality of the model by 12%. In the second model, for improving the semantic quality of the sentences generated by the sequence-to-sequence models and also to better learn the dependencies between words, we use dynamic memory networks. This memory is used to hold the information that led to the generation of the current word sequence. In this way, when generating a new word in the output, the model will pay more attention to the previously generated words and ex- pressed information. This prevents the occurrence of semantic defects such as the repeated expression of information or the expression of redundant information. This model is able to reduce the semantic defects of the generated sentences by 20%. In the third model, we used the base Transformer network to learn longer dependencies between words. In order to make the transformer network more compatible with the purpose of generating text from data, we applied changes to the self-attention layers of the encoder and decoder. Then we used it as a generative network in the framework of adversarial learning. The discriminative network in this model is a Triplet network including encoder blocks of the base transformer. We de- fined the training objective function of this model in such a way that in the training process, the similarity of the generated sentence with the reference sentence is maintained, both at the word level and at the meaning level. Compared to the base transformer and pre-trained language models, the enhanced transformer network is able to reduce the semantic defects of the sentences by an average of 11%. In addition, the sentences generated by the proposed model have higher linguistic quality and diversity compared to the baseline models
- Keywords:
- Recurrent Neural Networks ; Natural Language Generation System ; Data-to-Text System ; Generative Adversarial Networks ; Dependency Information ; Transformer Network
Digital Object List-
محتواي کتاب - view
Bookmark