Loading...

Evaluation of NeuroEvolution of Augmenting Topologies in Cooperative Multi-Agent Learning

Iravanian, Sina | 2011

941 Viewed
  1. Type of Document: M.Sc. Thesis
  2. Language: Farsi
  3. Document No: 41780 (02)
  4. University: Sharif University of Technology
  5. Department: Mathematical Sciences
  6. Advisor(s): Mahdavi Amiri, Nezameddin; Beigy, Hamid
  7. Abstract:
  8. In multi-agent systems (MAS), collective behavior of autonomous agents and complexities arisen by their interactions are studied, while they are exploited to solve real-world complex problems. Machine learning methods are frequently used for problem solving in MAS, because complexities in these systems prevent a programmer to thoroughly describe the agents’ behaviors and the rules governing them.Reinforcement learning (RL) is one of the most commonly used learning methods for intelligent agents,because it does not need a model of the environment and learns agents’ policies through trial and error.Conventional RL algorithms store and update utilities for every possible state in a table. One condition for the RL algorithms to converge is that all states be visited infinitely often. Satisfying this condition and in general storing such a table is not possible for large or continuous state spaces, especially in multi-agent systems where the size of state space grows exponentially with the number of agents. For this reason, an approximation of the real table is often maintained. In this thesis, application of a family of methods called NeuroEvolution of Augmenting Topologies (NEAT) to cooperative MAS in which non-communicating agents decide independently is studied and evaluated. The evolved neural networks are used as function approximators in the agents’ RL algorithms. The topology and connection weights of the neural networks are evolved through NEAT. The algorithms are evaluated in two test-beds: predator-prey and grid-world soccer. Empirical results in the predator-prey environment convey that neural network controllers evolved by NEAT, through cooperative co-evolutionary learning, reaches the optimal policy faster than other methods, while HyperNEAT team learning with 3D substrate demonstrates a more reliable team behavior in on-line scenarios. Two methods are proposed in the grid-world soccer environment, which exploit geometrical properties of the game to learn team strategies. One method, which makes use of a 4D substrate to represent team strategies, learns the optimal team policies in a short time, and scores a significantly higher goal difference. Another interesting advantage of the proposed method is that it can be scaled to a larger environment, more players, and different team formations with no further learning required
  9. Keywords:
  10. Multiagent System ; Learning Coordination ; Multi-Agent Reinforcement Learning ; Neuro-Evolutionary Methods

 Digital Object List

 Bookmark

...see more