Fault-Tolerant Federated Reinforcement Learning with Theoretical Guarantee

The growing literature of Federated Learning (FL) has recently inspired Federated

Reinforcement Learning (FRL) to encourage multiple agents to federatively build

a better decision-making policy without sharing raw trajectories. Despite its

promising applications, existing works on FRL fail to I) provide theoretical analysis

on its convergence, and II) account for random system failures and adversarial

attacks. Towards this end, we propose the first FRL framework the convergence of

which is guaranteed and tolerant to less than half of the participating agents being

random system failures or adversarial attackers. We prove that the sample efficiency

of the proposed framework is guaranteed to improve with the number of agents

and is able to account for such potential failures or attacks. All theoretical results

are empirically verified on various RL benchmark tasks. Our code is available at

https://github.com/flint-xf-fan/Byzantine-Federeated-RL.