Loading...

Neuromuscular control of the point to point and oscillatory movements of a sagittal arm with the actor-critic reinforcement learning method

Golkhou, V ; Sharif University of Technology | 2005

646 Viewed
  1. Type of Document: Article
  2. DOI: 10.1080/10255840500167952
  3. Publisher: 2005
  4. Abstract:
  5. In this study, we have used a single link system with a pair of muscles that are excited with alpha and gamma signals to achieve both point to point and oscillatory movements with variable amplitude and frequency. The system is highly nonlinear in all its physical and physiological attributes. The major physiological characteristics of this system are simultaneous activation of a pair of nonlinear musclelike- actuators for control purposes, existence of nonlinear spindle-like sensors and Golgi tendon organlike sensor, actions of gravity and external loading. Transmission delays are included in the afferent and efferent neural paths to account for a more accurate representation of the reflex loops. A reinforcement learning method with an actor-critic (AC) architecture instead of middle and low level of central nervous system (CNS), is used to track a desired trajectory. The actor in this structure is a two layer feedforward neural network and the critic is a model of the cerebellum. The critic is trained by state-action-reward-state-action (SARSA) method. The critic will train the actor by supervisory learning based on the prior experiences. Simulation studies of oscillatory movements based on the proposed algorithm demonstrate excellent tracking capability and after 280 epochs the RMS error for position and velocity profiles were 0.02, 0.04 rad and rad/s, respectively. © 2005 Taylor & Francis Ltd
  6. Keywords:
  7. Actor critic ; Actor-critic reinforcement learning ; Central nervous systems ; CMAC ; Control purpose ; External loading ; Gamma signals ; Low level ; Motor control ; Neuromuscular control ; Oscillatory movements ; Physiological characteristics ; Point to point ; Prior experience ; Reinforcement learning method ; RMS errors ; Simulation ; Simulation studies ; Single link ; Tracking capability ; Transmission delays ; Variable amplitudes ; Velocity profiles ; AC generator motors ; Feedforward neural networks ; Loading ; Physiology ; Sensors ; Reinforcement learning ; Artificial neural network ; Biological model ; Biological rhythm ; Body equilibrium ; Computer simulation ; Evaluation ; Feedback system ; Human ; Innervation ; Movement (physiology) ; Muscle contraction ; Myotatic reflex ; Reinforcement ; Skeletal muscle ; Arm ; Biological Clocks ; Computer Simulation ; Feedback ; Humans ; Models, Neurological ; Movement ; Muscle, Skeletal ; Musculoskeletal Equilibrium ; Neural Networks (Computer) ; Reflex, Stretch ; Reinforcement (Psychology)
  8. Source: Computer Methods in Biomechanics and Biomedical Engineering ; Volume 8, Issue 2 , 2005 , Pages 103-113 ; 10255842 (ISSN)
  9. URL: https://www.tandfonline.com/doi/abs/10.1080/10255840500167952