Portal de Periódicos da CAPES

Distributed Neural Networks Training for Robotic Manipulation With Consensus Algorithm

2022; Institute of Electrical and Electronics Engineers; Volume: 35; Issue: 2 Linguagem: Inglês

10.1109/tnnls.2022.3191021

2162-2388

Wenxing Liu, Hanlin Niu, Inmo Jang, Guido Herrmann, Joaquín Carrasco,

Adaptive Dynamic Programming Control

In this article, we propose an algorithm that combines actor-critic-based off-policy method with consensus-based distributed training to deal with multiagent deep reinforcement learning problems. Specifically, convergence analysis of a consensus algorithm for a type of nonlinear system with a Lyapunov method is developed, and we use this result to analyze the convergence properties of the actor training parameters and the critic training parameters in our algorithm. Through the convergence analysis, it can be verified that all agents will converge to the same optimal model as the training time goes to infinity. To validate the implementation of our algorithm, a multiagent training framework is proposed to train each Universal Robot 5 (UR5) robot arm to reach the random target position. Finally, experiments are provided to demonstrate the effectiveness and feasibility of the proposed algorithm.