Loading...

Application of actor-critic reinforcement learning method for control of a sagittal arm during oscillatory movement

Golkhou, V ; Sharif University of Technology | 2004

114 Viewed
  1. Type of Document: Article
  2. DOI: 10.4015/s1016237204000426
  3. Publisher: Institute of Biomedical Engineering , 2004
  4. Abstract:
  5. Numerous disciplines are engaged in studies involving motor control. In this study, we have used a single link system with a pair of muscles that are excited with alpha and gamma signals to achieve an oscillatory movement with variable amplitude and frequency. The system is highly nonlinear in all its physical and physiological attributes. The major physiological characteristics of this system are simultaneous activation of a pair of nonlinear muscle-like-actuators for control purposes, existence of nonlinear spindle-like sensors and Golgi tendon organ-like sensor, actions of gravity and external loading. Transmission delays are included in the afferent and efferent neural paths to account for a more accurate representation of the reflex loops. The profile of excitation is difficult to predict a priori, hence we have used a reinforcement learning approach to track a desired trajectory. This paper proposes a reinforcement learning method with an Actor-Critic architecture instead of middle and low level of central nervous system (CNS). The Actor in this structure is a two layer feedforward neural network and the Critic is a model of the cerebellum. The Critic is trained by State-Action-Reward-State- Action (SARSA) method. The Critic will train the Actor by supervisory learning based on previous experiences. The system is implemented on a PC using Matlab® and Simulink® Software. To enhance the computational performance a number of C codes were also written. The effectiveness and the biological plausibility of the present model are demonstrated by several simulations. The system showed excellent tracking capability and after 280 epochs the RMS error for position and velocity profiles were 0.02, 0.04 radian and radian/sec, respectively
  6. Keywords:
  7. Actor-critic ; CMAC ; Motor control ; Reinforcement learning ; Simulink
  8. Source: Biomedical Engineering - Applications, Basis and Communications ; Volume 16, Issue 6 , 2004 , Pages 305-312 ; 10162372 (ISSN)
  9. URL: https://www.worldscientific.com/doi/10.4015/S1016237204000426