A model on the ‘Pins Face Recognition’ dataset was trained. This dataset is a collection of 17,534 celebrity faces of 105 different celebrities. These images have been originally collected from Pinterest through a scrapper-bot made by Python and Selenium. The images in this dataset comprise faces cropped using dlib.
 
Transfer Learning was implemented while training the model. The MobileNetV2 model and imagenet weights were used. The MobileNetV2 model used is a pre-trained model that consists of 155 layers. The imagenet weights were retained to get better accuracy on the samples. Imagenet is a dataset that consists of more than 14 million images separated into more than 21,000 different categories. Data augmentation was implemented so as to reduce the overfitting of the model. The performance of the model was evaluated by observing the validation set accuracy and the confusion matrix generated from the validation data. 88% accuracy was observed.
Adversarial Attacks were conducted against the model using the FGSM(Fast Gradient Sign Method). The attacks were L infinity, or L1, or L2 norm bounded. The attacks were also differentiated into targeted and untargeted attacks. Positive results were seen by analyzing and comparing the confusion matrices of pre- and post-attack data. Hence, the model was determined to be adequate for examining the effects of an adversarial attack via facial attribute addition.
 
Grad-CAM was implemented, which resulted in darkening/hiding the most crucial parts of the face for celebrity classification by the model. Significant misclassification results were observed.
 
Facial Attribute Addition was implemented using Dlib’s 68-point facial landmark detector. An attribute in the form of a small colored cover was added to specific parts of the face. Significant misclassification results were observed, which translated to the successful completion of the project.

Pins Face Recognition Dataset: https://www.kaggle.com/datasets/hereisburak/pins-face-recognition
Dlib's 68-point landmark model: https://www.kaggle.com/datasets/tejaschandakkar/models