Loading...

Integrated Guidance and Control of a Hexacopter Equipped with Robotic Arm using Reinforcement Learning

Shobeiry, Mohammad Mahdi | 2025

0 Viewed
  1. Type of Document: M.Sc. Thesis
  2. Language: Farsi
  3. Document No: 58386 (45)
  4. University: Sharif University of Technology
  5. Department: Aerospace Engineering
  6. Advisor(s): Emami Khansari, Mohammad Ali
  7. Abstract:
  8. The primary objective of this research is the integrated guidance and control of a hexacopter system equipped with a robotic arm using deep reinforcement learning algorithms in an end-to-end and model-free framework. The hexacopter with a robotic arm system possesses 6 degrees of freedom for the hexacopter and 2 degrees of freedom for the robotic arm, which consists of rigid links attached to the underside of the hexacopter. The dynamic model of this system has been developed in an integrated manner to properly account for the coupling effects between the equations of the hexacopter and the arm in the analyses. After developing the equations describing the system's behavior, the goal is to control all its degrees of freedom using reinforcement learning. The mission defined for this system is a point-tracking mission: the system must reach the location of an external object while maintaining stability, then use its 2 DoF robotic arm to pick up this object with unknown characteristics from a surface, and finally, while maintaining the stability of the combined hexacopter-with-robotic-arm and external object, deliver the object to the desired position. To succeed in this mission, defining a multi-objective reward function is essential for the agent's success. This function includes guidance of the hexacopter flight path, guidance of the robotic arm's path, control of system’s states and the system's control inputs. To evaluate the robustness of the implemented reinforcement learning algorithms, the effects of model uncertainty, disturbances, actuator fault, and noise on the system's performance and mission success are studied. The reinforcement learning algorithms used in this research are model-free, which increases the complexity of the design process due to their success in controlling a complex dynamic system with significant coupling effects between the hexacopter and the arm. According to this research, a comparison of three reinforcement learning algorithms—SAC, TD3, and PPO—with a classical PID controller concludes the superiority of the reinforcement learning algorithms in terms of better handling the coupling effects between the hexacopter and the arm, successfully completing the mission in a shorter time, and compensating for the effects of uncertainties, disturbances, and failures. Among these reinforcement learning algorithms, the TD3 algorithm has the highest mission execution speed, the PPO algorithm has lower speed but the highest robustness and less control effort. The SAC algorithm also shows good speed in completing the mission but has less robustness in challenging conditions compared to TD3 and PPO. The outcome of this research paves the way for designing integrated guidance and control structures for complex dynamic systems in the field of aerial manipulation
  9. Keywords:
  10. Deep Reinforcement Learning ; End-to-End Learning ; Integrated Guidance and Control (IGC) ; Multirotor Control System ; Waypoint Tracking ; End-to-End Reinforcement Learning ; Multirotor with Robotic Arm ; Aerial Manipulation

 Digital Object List

 Bookmark

...see more