Portal de Periódicos da CAPES

Forgery face detection via adaptive learning from multiple experts

2023; Elsevier BV; Volume: 527; Linguagem: Inglês

10.1016/j.neucom.2023.01.017

1872-8286

Xinghe Fu, Shengming Li, Yike Yuan, Bin Li, Xi Li,

Generative Adversarial Networks and Image Synthesis

As an important and challenging problem, Face Forgery Detection has gained considerable attention. Usually, it suffers from the diversity of forgery patterns in forgery images, which requires a detection model to have capability of capturing various patterns in the challenging scenarios. To address this problem, we present a divide-and-aggregate learning framework to build multi-expert models and integrate them into a unified model. Firstly, the built multi-expert models are pre-trained to capture and preserve the specific forgery pattern produced by each manipulation method separately. Secondly, to transfer diverse knowledge of experts, we propose an integrating approach based on knowledge distillation. However, the difference of manipulation-aware knowledge among these experts concerns the way of distillation when the knowledge is combined in the only student model. Thus, to determine the importance of each expert, we propose a sample-aware Adaptive Learning from Experts strategy (ALFE) to assign adaptive expert distillation weights for each fake sample based on the predictions of each expert. Experiments show that our method achieves SOTA performances on ACC/AUC in the benchmark of FaceForensics++, demonstrating the effectiveness of our proposed method.