Loading...

Incremental Representative Words Extraction of Persian Weblogs with Change of Theme Detection Using Graph Approach

Sayyadi, Mohsen | 2011

401 Viewed
  1. Type of Document: M.Sc. Thesis
  2. Language: Farsi
  3. Document No: 42599 (19)
  4. University: Sharif University of Technology
  5. Department: Computer Enginnering
  6. Advisor(s): hodsi, Mohammad
  7. Abstract:
  8. Although dimension reduction techniques for text documents can be used for preprocessing of blogs, these techniques will be more effective if they deal with the nature of the blogs properly. In this project we propose a novel algorithm called PostRank using shallow approach to identify theme of the blog or blog representative words in order to reduce the dimensions of blogs. PostRank uses a graph-based syntactic representation of the weblog by taking into account some structural features of weblog. At the first step it models the blog as a complete graph and assumes the theme of the blog as a query applied to a search engine like Google and each post as a search result. It tries to rank the posts using markov chain model like PageRank in Google. We used the ranking model under the assumption that top ranked nodes contain blog best representative words. Then it tries to identify post groups according to their scores. Finally this algorithm analyzes the first group using statistical methods(like TF-IDF) to identify blog representative words. Other groups are candidates of having blog theme after occurring change of theme to the blog. By arriving new instances of posts we try to update the blog graph by setting the initial scores of old nodes in the markov chain to their final score from last run and continue the PostRank iterations until reaching convergence point. If half of the representative words have changed we would say that theme of the weblog has been changed. We evaluated our method on the Persianblog dataset and obtained promising results. The blogs have been assigned to ten representative words by human beings and the results of PostRank have been compared to them and results of old related algorithms in this area
  9. Keywords:
  10. Dimension Reduction ; Weblog ; Ranking ; Markov Chain ; Blog Theme ; Representative Words

 Digital Object List

 Bookmark

No TOC