Portal de Periódicos da CAPES

Amata: An Annealing Mechanism for Adversarial Training Acceleration

2021; Association for the Advancement of Artificial Intelligence; Volume: 35; Issue: 12 Linguagem: Inglês

10.1609/aaai.v35i12.17278

2374-3468

Nanyang Ye, Qianxiao Li, Xiao-Yun Zhou, Zhanxing Zhu,

Anomaly Detection Techniques and Applications

Despite the empirical success in various domains, it has been revealed that deep neural networks are vulnerable to maliciously perturbed input data that much degrade their performance. This is known as adversarial attacks. To counter adversarial attacks, adversarial training formulated as a form of robust optimization has been demonstrated to be effective. However, conducting adversarial training brings much computational overhead compared with standard training. In order to reduce the computational cost, we propose an annealing mechanism, Amata, to reduce the overhead associated with adversarial training. The proposed Amata is provably convergent, well-motivated from the lens of optimal control theory and can be combined with existing acceleration methods to further enhance performance. It is demonstrated that on standard datasets, Amata can achieve similar or better robustness with around 1/3 to 1/2 the computational time compared with traditional methods. In addition, Amata can be incorporated into other adversarial training acceleration algorithms (e.g. YOPO, Free, Fast, and ATTA), which leads to further reduction in computational time on large-scale problems.