论文标题
标志：对盲人和拜占庭对手的缺陷耐受性
SignSGD: Fault-Tolerance to Blind and Byzantine Adversaries
论文作者
论文摘要
通过在几种设备之间共享计算，分布式学习已成为培训不断增长的模型的必要性。但是，某些设备可能是故意的，无论是否有故障，可以防止适当的收敛。事实上，基线分布的SGD算法不会在一个拜占庭对手的情况下收敛。在本文中，我们关注从SGD得出的更健壮的符号算法。我们为标志的收敛速率提供了上限，证明了该新版本对拜占庭对手的强大。我们与试图粉碎学习过程的拜占庭策略一起实施了标志。因此，我们提供了实验中的经验观察以支持我们的理论。我们的代码可在github https://github.com/jasonakoun/signsgd-fault-tolerance上获得，我们的实验可通过使用提供的参数来重现。
Distributed learning has become a necessity for training ever-growing models by sharing calculation among several devices. However, some of the devices can be faulty, deliberately or not, preventing the proper convergence. As a matter of fact, the baseline distributed SGD algorithm does not converge in the presence of one Byzantine adversary. In this article we focus on the more robust SignSGD algorithm derived from SGD. We provide an upper bound for the convergence rate of SignSGD proving that this new version is robust to Byzantine adversaries. We implemented SignSGD along with Byzantine strategies attempting to crush the learning process. Therefore, we provide empirical observations from our experiments to support our theory. Our code is available on GitHub https://github.com/jasonakoun/signsgd-fault-tolerance and our experiments are reproducible by using the provided parameters.
