Face recognition using Caffe - Help please!

[Pradeep]

Has anyone used Caffe to do face detection /recognition? Is there any step-by-step guide on how to go about it? 

There are many frameworks that are available just for Face rec, but is it possible to train Caffe to do Face rec while also having the capability to recognize other objects. 

how to do the training for this.? 

thanks

[eran paz]

In general you have 2 face recognition schemes:

1. same \ not same

2. person identification.

For same \ not same:

Each instance is comprised of 2 facial images and your model should return whether it's the same person in both images or not.

A common method for implementing this (using caffe \ other deep learning framework) is the siamese network: http://caffe.berkeleyvision.org/gathered/examples/siamese.html

Advantages:

From a relatively small dataset (e.g. lfw) you can create huge datasets for training by pairing images on your own

given 2 images, you can get a same \ not same classification for persons outside of your training dataset

Disadvantages:

In order to get an actual identification you need to compare to every person in your reference database (you might get few "same" results)

For person identification:

Each instance is an image and you want to classify to the correct identity.

The number of classes is the same as the number of persons in your data.

You can use a "regular" CNN for this type of classification.

Advantages:

(Assuming softmax output) you get the probability that a new image is the same to someone in your reference database, so you can check how close it is to "second place"

Network structure is simpler (although the siamese architecture is not really that complicated).

Disadvantages:

A lot harder to train a classifier for K classes than 2

Your model is limited to finding an identity within your reference database (you can't classify a new person)

All of the above doesn't describe any pre-processing you might want to have on facial images.

It's debatable how much this pre-processing is required with deep learning models, but they are practically a "must" for non deep learning (and I believe they improve deep learning as well).

Some of the pre-processing might include:

Facial landmark detection

Face alignment (2D) or Frontalization (3D)

Image cropping

and more...

A good place to start in my opinion is LFW (Labeled Faces in the Wild) which gives a relatively large dataset (~13.5K good quality images)

http://vis-www.cs.umass.edu/lfw/

[StevenL]

Thanks Eran:

Actually I am using exactly 2nd way (person identification) you mentioned in my approach for face recognition, I notice that LFW has already provide an funneled pictures which have been aligned. Have you done a prototype by using caffe in face recognition?

BR

[Axel Angel]

I was wondering an other method: Imagine you can create a fingerprint of a person using a network, then using a distance function to compare to known people. That could serve to identify new people as well (create a new entry for this fingerprint). If you think about it, it doesn't depend on the content at all, it could be to identify objects (cars, and such) as well.

I'm wondering how such method can be implemented: Can we take a siamese network or auto-encoders and take the output of a certain layer, for siamese it could be just before the merging layer, for auto-encoder that could be the smallest layer (in the middle). Then how would it perform? Are there papers which tried such method?

People in the litterature are probably calling this: feature extraction, unique signatures. Any thoughts?

[Yiak Wang Yi]

Hi, eran:

I am working on identity detection, but my problem is that due to limitation of my computation power, I have to use a pretrained model to do identity prediction.

So I need to figure out which caffemodel I can use to detect an image where figures inside come from LFW.

Also I need to do corresponding image alignment so that the image is well accepted by the constructed caffe net.

I have gone through caffe-face to checkout model, but found scholars said that we trained a model using other dataset, and achieved good performance in LFW, you can check their paper!  I downloaded face_model from "caffe-face", but I found the last layer is "fc"! I need "softmax"!!!. 

Hence the caffemodel parameter not useful for me! They are onsense to me!

I am seeking your help to tell which caffemodel should I use?

在 2015年3月9日星期一 UTC+8下午1:55:53，eran paz写道：

[Jonathan R. Williford]

Hi Yiak,

Can you give me more details on how you want to apply the model to the LFW dataset? Are you going to make it a siamese network and output whether two images are the same or different people? Or are you going to make it so that the number of classes are the number of people in the dataset?

I didn't see the pre-trained version of caffe-face. How many channels does the last output give?

Cheers,

Jonathan

--
You received this message because you are subscribed to the Google Groups "Caffe Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to caffe-users+unsubscribe@googlegroups.com.
To post to this group, send email to caffe...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/caffe-users/d35995d8-8f1c-40b8-bb80-c671a72e84b9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jonathan R. Williford]

You could freeze the weights of the model and add another FC layer to get the correct number of channels and then a softmax(withloss) layer? Training a single learnable layer should be feasible on a CPU. You have to really make sure that all the weights are frozen though. Of course, using some other metric outside of the network would also work (and might be faster).

Cheers,

Jonathan

On Thu, Aug 24, 2017 at 9:11 AM, Yiak Wang Yi <yia...@gmail.com> wrote:

Hi:

Problem: give an arbitrary image (3*250*250)(image alignment for the model needed), output faces inside the image. Hence this is an identity detection problem.

I want to use an exsiting model:

caffemodel + deploy.prototxt

to detect an image like this:

net = caffe.Net(model_def,

                model_weights,

                caffe.TEST)

...

out = net.forward()

Idealy, output should be from softmax  to tell me which label the face is, and I can use some file like lfw_attribute.txt to query the name of the face.

If there is no public model for that, an feature output like 'fc7' are accpeted, but I need to use that to get the person: e.g. loop through lfw to get nearest images in some metrics. But his method is not recommended, because I have to compute all the image feature vectors: O(n*FORWARD), my computer **might not** able to do that.

For both of the methods, I need info:

1. image alignment for input image

2. net.data tranform procesure

3. label files corresponding to the model weights