论文标题

GLFF:AI合成图像检测的全球和局部特征融合

GLFF: Global and Local Feature Fusion for AI-synthesized Image Detection

论文作者

Ju, Yan, Jia, Shan, Cai, Jialing, Guan, Haiying, Lyu, Siwei

论文摘要

随着深层生成模型的快速发展(例如生成对抗网络和扩散模型),AI合成的图像现在具有如此高质量,以至于人类几乎无法将它们与原始图像区分开。尽管现有的检测方法在特定的评估设置中表现出了高性能,例如,在看到的模型或没有现实世界后处理的图像上,它们倾向于在现实情况下遭受严重的性能退化,在现实情况下,可以通过更强大的生成模型或结合各种后处理操作来生成测试图像。为了解决这个问题,我们提出了一个全局和局部特征融合(GLFF)框架,通过将整个图像中的多尺度全局特征与精致的本地特征相结合,以了解AI综合图像检测的信息,以学习丰富和歧视性表示。 GLFF融合了来自两个分支的信息:全球分支,用于提取多尺度语义特征和本地分支,以选择信息丰富的补丁,以详细的本地伪像提取。由于缺乏综合图像数据集模拟现实世界中的评估应用程序,我们进一步创建了一个具有挑战性的假图像数据集,名为DeepFakefaceForensics(DF 3),该数据集包含6个最先进的一代模型和各种后处理技术,以接近现实情况。实验结果证明了我们方法对拟议的DF 3数据集和其他三个开源数据集的优越性。

With the rapid development of deep generative models (such as Generative Adversarial Networks and Diffusion models), AI-synthesized images are now of such high quality that humans can hardly distinguish them from pristine ones. Although existing detection methods have shown high performance in specific evaluation settings, e.g., on images from seen models or on images without real-world post-processing, they tend to suffer serious performance degradation in real-world scenarios where testing images can be generated by more powerful generation models or combined with various post-processing operations. To address this issue, we propose a Global and Local Feature Fusion (GLFF) framework to learn rich and discriminative representations by combining multi-scale global features from the whole image with refined local features from informative patches for AI synthesized image detection. GLFF fuses information from two branches: the global branch to extract multi-scale semantic features and the local branch to select informative patches for detailed local artifacts extraction. Due to the lack of a synthesized image dataset simulating real-world applications for evaluation, we further create a challenging fake image dataset, named DeepFakeFaceForensics (DF 3 ), which contains 6 state-of-the-art generation models and a variety of post-processing techniques to approach the real-world scenarios. Experimental results demonstrate the superiority of our method to the state-of-the-art methods on the proposed DF 3 dataset and three other open-source datasets.

扫码加入交流群

加入微信交流群

微信交流群二维码

扫码加入学术交流群,获取更多资源