Visible to the public Detecting Deepfakes with Metric Learning

TitleDetecting Deepfakes with Metric Learning
Publication TypeConference Paper
Year of Publication2020
AuthorsKumar, A., Bhavsar, A., Verma, R.
Conference Name2020 8th International Workshop on Biometrics and Forensics (IWBF)
Date Publishedapr
Keywordsdata compression, deep learning approaches, DeepFake, deepfake detection, Deepfakes, deepfakes classification, digital media content, face-swapping applications, FaceApp, FaceBlender, fake videos, feature extraction, feature space distance, Human Behavior, human factors, image classification, image texture, learning (artificial intelligence), metric learning approach, Metrics, MixBooth, neural texture dataset, pubcrawl, resilience, Resiliency, Scalability, Snapchat, social media platforms, social networking (online), Triplet Network, triplet network architecture, Video Forensics
AbstractWith the arrival of several face-swapping applications such as FaceApp, SnapChat, MixBooth, FaceBlender and many more, the authenticity of digital media content is hanging on a very loose thread. On social media platforms, videos are widely circulated often at a high compression factor. In this work, we analyze several deep learning approaches in the context of deepfakes classification in high compression scenarios and demonstrate that a proposed approach based on metric learning can be very effective in performing such a classification. Using less number of frames per video to assess its realism, the metric learning approach using a triplet network architecture proves to be fruitful. It learns to enhance the feature space distance between the cluster of real and fake videos embedding vectors. We validated our approaches on two datasets to analyze the behavior in different environments. We achieved a state-of-the-art AUC score of 99.2% on the Celeb-DF dataset and accuracy of 90.71% on a highly compressed Neural Texture dataset. Our approach is especially helpful on social media platforms where data compression is inevitable.
Citation Keykumar_detecting_2020