Loading...

Persian language modeling using recurrent neural networks

Hosseini Saravani, H ; Sharif University of Technology | 2019

389 Viewed
  1. Type of Document: Article
  2. DOI: 10.1109/ISTEL.2018.8661032
  3. Publisher: Institute of Electrical and Electronics Engineers Inc , 2019
  4. Abstract:
  5. In this paper, recurrent neural networks are applied to language modeling of Persian, using word embedding as word representation. To this aim, unidirectional and bidirectional Long Short-Term Memory (LSTM) networks are used, and the perplexity of Persian language models on a 100-million-word data set is evaluated. The effect of various parameters, including number of hidden layers and size of LSTM units, on the performance of the networks in reducing the perplexity of the models are investigated. Among different LSTM language models, the best perplexity, which is equal to 59.05, is achieved from a 2-layer bidirectional LSTM model. Comparing this value with the perplexity of the classical Trigram model, which is equal to 138, an improvement in the modeling is noticeable, which is due to the ability of neural networks to make a higher generalization in comparison with the well-known N-gram model. © 2018 IEEE
  6. Keywords:
  7. N-gram ; Neural network ; Persian language modeling ; Computational linguistics ; Embeddings ; Modeling languages ; Neural networks ; Hidden layers ; Language model ; LSTM ; N-gram modeling ; N-grams ; Persian languages ; Word embedding ; Word representations ; Long short-term memory
  8. Source: 9th International Symposium on Telecommunication, IST 2018, 17 December 2018 through 19 December 2018 ; 2019 , Pages 207-210 ; 9781538682746 (ISBN)
  9. URL: https://ieeexplore.ieee.org/document/8661032