Loading...

Recurrent Neural Network Language Modeling For Persian

Hosseini Saravani, Habib | 2018

654 Viewed
  1. Type of Document: M.Sc. Thesis
  2. Language: Farsi
  3. Document No: 50774 (31)
  4. University: Sharif University of Technology
  5. Department: Languages and Linguistics Center
  6. Advisor(s): Bahrani, Mohammad; Veisi, Hadi
  7. Abstract:
  8. Neural Networks have been applied to Language Modeling to solve a major problem that N-gram language models could not overcome: discreteness of the words. Generally, neural networks were successful in solving this problem and improved Language Modeling by reducing the perplexity of the models. Neural networks can find grammatical and semantic connections among the words using word embedding which maps each word to a low dimensional feature vector of real numbers. In this research, different kinds of neural network applied to Language Modeling has been reviewed. Also, it has been tried to reduce the perplexity of Persian language models on a 100-million scale data set using a single-layer LSTM, a 2-layer LSTM, a single-layer bidirectional LSTM and a 2-layer bidirectional LSTM neural network. We use two single-layer and also 2-layer networks to investigate the effect of number of hidden layers on a 100-million scale language modeling, and, based on the scale of the data set we use, we try to investigate the effect of the size of LSTM layer(s) on the performance of the network in reducing the perplexity of the models. In this paper, comparing the results of LSTM language models with an Tri-gram language model, we show that LSTM neural networks can be used to reduce the perplexity of language models. We also show that using multi-layer LSTM neural networks does not necessarily lead to a better performance in reducing the perplexity of 100-million scale language models; however, in most of the models we trained using 2-layer LSTMs, there was, on average, a 1.6 percent decrease in the perplexity of the models. Finally, we show that bidirectional LSTM neural networks can improve the performance of unidirectional LSTM neural networks in language modeling, reducing the perplexity of unidirectional LSTM language models, on average, by 4 percent
  9. Keywords:
  10. Persian Language ; Bidirectional ; Language Modeling ; Neural Network ; N-Gram Language Model ; Long Short Term Memory (LSTM)

 Digital Object List

 Bookmark

No TOC