A novel graphical approach to automatic abstraction in reinforcement learning

Taghizadeh, N ; Sharif University of Technology | 2013

662 Viewed
  1. Type of Document: Article
  2. DOI: 10.1016/j.robot.2013.04.010
  3. Publisher: 2013
  4. Abstract:
  5. Recent researches on automatic skill acquisition in reinforcement learning have focused on subgoal discovery methods. Among them, algorithms based on graph partitioning have achieved higher performance. In this paper, we propose a new automatic skill acquisition framework based on graph partitioning approach. The main steps of this framework are identifying subgoals and discovering useful skills. We propose two subgoal discovery algorithms, which use spectral analysis on the transition graph of the learning agent. The first proposed algorithm, incorporates k′-means algorithm with spectral clustering. In the second algorithm, eigenvector centrality measure is utilized and options are discovered. Moreover, we propose an algorithm for pruning useless options, which cause additional costs for the learning agent. The experimental results on various problems show significant improvement in the learning performance of the agent
  6. Keywords:
  7. Automatic skill acquisition ; Eigenvector centrality ; Option pruning ; Spectral graph clustering ; Automatic abstractions ; Discovery algorithm ; Eigenvector centralities ; Learning performance ; Skill acquisition ; Spectral clustering ; Eigenvalues and eigenfunctions ; Graph theory ; Mergers and acquisitions ; Reinforcement learning ; Spectrum analysis ; Learning algorithms
  8. Source: Robotics and Autonomous Systems ; Volume 61, Issue 8 , 2013 , Pages 821-835 ; 09218890 (ISSN)
  9. URL: http://www.sciencedirect.com/science/article/pii/S0921889013000729