Loading...

Cooperative Mission of Autonomous Underwater Robots Using Reinforcement Learning

Ahmadi, Fardin | 2024

0 Viewed
  1. Type of Document: M.Sc. Thesis
  2. Language: Farsi
  3. Document No: 57605 (08)
  4. University: Sharif University of Technology
  5. Department: Mechanical Engineering
  6. Advisor(s): Sayyaadi, Hassan; Hoviattalab, Maryam
  7. Abstract:
  8. In recent years, the development and optimization of Autonomous Underwater Vehicles (AUVs) have become crucial in fields such as marine science and rescue operations, owing to their extensive applications. However, challenges such as improving the stability and coordination of AUV fleets remain. This study aims to simulate and evaluate the performance of two advanced reinforcement learning algorithms, namely Twin Delayed Deep Deterministic Policy Gradient (TD3) and Soft Actor-Critic (SAC), to optimize the behavior and control of an AUV fleet. To achieve this, we employed the Python Vehicle Simulator and the Gymnasium library within the Stable Baselines3 framework. The findings indicate that TD3 outperforms SAC, particularly in terms of increasing the average episodic reward and reducing variability, thereby providing a more stable policy. TD3's enhanced stability is attributed to techniques such as clipped double-Q learning, delayed policy updates, and target policy smoothing, yielding favorable results in AUV coordination and control. Conversely, while SAC's entropy optimization facilitates greater exploration and prevents premature convergence to local optima, it exhibits somewhat less consistent results than TD3 in specific scenarios. Limitations of this study include simulation environment constraints and sensitivity to hyperparameter settings, which require further fine-tuning for enhanced performance. For future research, it is recommended to explore advanced reinforcement learning algorithms, leverage automated hyperparameter optimization, and evaluate AUV performance in real-world settings to gain a more comprehensive understanding and improve the capabilities of these vehicles
  9. Keywords:
  10. Autonomous Underwater Vehicle ; Reinforcement Learning ; Numerical Simulation ; Soft Actor Critic (SAC)Algorithm ; Twin Delayed Deep Deterministic Policy Gradient (TD3)Algorithm ; UnderWater Robot

 Digital Object List

 Bookmark

No TOC