[ML-news] [Software] Code Release: FaceDancer: Pose- and Occlusion-Aware High Fidelity Face Swapping

[Eren Erdal Aksoy]

We release the code related to our new upcoming IEEE/CVF WACV 2023 paper “FaceDancer: Pose- and Occlusion-Aware High Fidelity Face Swapping” 

This is a new face swapping framework which is fast, simple, subject-agnostic and works consistently on videos

You can already play around with it over at Hugging Face: https://lnkd.in/dmJQ-wh7

The source code is available at https://github.com/felixrosberg/FaceDancer

The paper preprint is also available at https://arxiv.org/abs/2210.10473

We propose  a new single-stage method for subject agnostic face swapping and identity transfer, named FaceDancer. We have two major contributions: Adaptive Feature Fusion Attention (AFFA) and Interpreted Feature Similarity Regularization (IFSR). 

• The AFFA module is embedded in the decoder and adaptively learns to fuse attribute features and features conditioned on identity information without requiring any additional facial segmentation process. 
• In IFSR, we leverage the intermediate features in an identity encoder to preserve important attributes such as head pose, facial expression, lighting, and occlusion in the target face, while still transferring the identity of the source face with high fidelity. 

We conduct extensive quantitative and qualitative experiments on various datasets and show that the proposed FaceDancer outperforms other state-of-the-art networks in terms of identity transfer, while having significantly better pose preservation than most of the previous methods. We hope that this work can inspire researchers in the field.

Best regards,

Eren Aksoy

---

Eren Erdal Aksoy 

Associate Professor

Halmstad University (HH)

School of Information Technology (ITE)

Center for Applied Intelligent Systems Research (CAISR)

P O Box 823, Room E526

Kristian IV:s väg 3

SE-301 18 Halmstad, Sweden

 

Mobile: +46 70 610 90 75

E-Mail: eren....@hh.se  

Web: https://aksoyeren.github.io/