Help with color FERET database

[daniel]

Hi,

I was able to successfully install Bob and run some experiments with some of the registered databases (such as The AT&T Database of Faces and the Labeled Faces in the Wild), generating ROC curves. Now I would like to do the same with a database that is not registered - the color FERET - but I can't figure out how to do so and would really appreciate if someone could provide some more detailed instructions.

[Manuel Günther]

Hi Daniel,

indeed, we have no default protocol or database package for the FERET dataset. Hence, you would need to create that on your own.

Fortunately, we have a generic file-list based database that is compatible with bob.bio. Maybe you can generate a database in this scheme: http://pythonhosted.org/bob.db.verification.filelist

Let us know in case of difficulties.

Best regards

Manuel

[daniel]

Hi Manuel, thanks for the reply.

Ok, so I have created the configuration file for the color FERET:

#!/usr/bin/env python

import facereclib

database = facereclib.databases.DatabaseFileList(
    database = bob.db.verification.filelist.Database(
        name = "colorferet"
        original_directory = "/home/CSU/work/colorferet",
        original_extension = ".pgm"
    ),
    name = 'colorferet',
)

and the directory:

color_feret/
├── dev
│   ├── for_models.lst
│   └── for_probes.lst
├── eval
│   ├── for_models.lst
│   └── for_probes.lst
└── norm
    └── train_world.lst

But I'm not even sure if that's correct. If so, what should I do next to register the database? Also, where should the configuration file colorferet.py be? It is currently in work/facereclib/configurations/databases where I found other similar configuration files.

Thank you for your help.

[Manuel Günther]

The database structure looks correct, though I don't see the contents of your ``for_models.lst`` and so on. Given that they are correct, the database should work like that.

Note, however, that the bob.db.verification.filelist.Database has different parameters in its constructor: http://pythonhosted.org/bob.db.verification.filelist/py_api.html#bob.db.verification.filelist.Database

Having just the configuration file is completely sufficient. You can pass the configuration file from where-ever it is (including its absolute or relative path) as a parameter to bin/faceverify.py. 

On the other hand, when you want to register the database as resource, please read this part of the documentation: http://pythonhosted.org/facereclib/contribute.html#adding-configuration-files

Please note that the facereclib is outdated, no longer maintained and superseded bu the bob.bio packages, see: http://pythonhosted.org/bob.bio.base/index.html

[daniel]

Manuel,

I changed the configuration file:

#!/usr/bin/env python

import bob.bio.base

database = bob.bio.base.database.DatabaseBob(
    database = bob.db.verification.filelist.Database(
    base_dir = "./color_feret",
        original_directory = "./colorferet2",

        original_extension = ".pgm"
    ),
    name = 'colorferet'

)

Then I tried to run bin/verify.py with the following command:

./bin/verify.py -p base -e eigenface -a lrpca -d /home/rsimmons/CSU/work/eggs/bob.bio.face-2.0.5-py2.7.egg/bob/bio/face/config/database/colorferet.py -s colorferet_test --verbose --verbose --verbose

After running for about 8 hours (after which i got a 'Killed' message on the terminal), it successfully generated a hdf5 file for each of the 11338 images from the database, under the temp/colorferet_test/preprocessed directory. However, this did not generate the scores-dev file, which I need to run bin/evaluate.py. What could cause this?

Thanks.

[Manuel Günther]

Daniel,

first of all, I am not sure if the combination of extractor and algorithm make any sense. the LRPCA algorithm requires its own type of preprocessor and extractor, any other combination is not tested or supported.

So, when running with ``-a lrpca`` you should also have the according ``-p lrpca`` and ``-e lrpca``. 

Note that the LRPCA algorithm works only on aligned images that have the correct size. This size is assured by the ``-p lrpca`` preprocessor, which will require annotated eye locations -- or detected ones (which might be a bit more complicated to set up, see here: https://gitlab.idiap.ch/bob/bob.bio.csu/blob/master/bob/bio/csu/test/test_preprocessors.py#L41 for an example; I hope that this version is already out there).

I guess that you had a memory issue, which has killed your process, which might be caused by the unusual combination of tools. Let me know in case of more trouble.

[daniel]

I didn't even notice i had the wrong options for the lrpca algorithm and can't even remeber why I changed it, thanks. However i just re-run the same command with the right options, and got the same result (no scores-dev file).

[Manuel Günther]

Did you make sure that the correct preprocessor and extractor actually ran and extracted new features (e.g., using the ``--force`` option, or specifying a new --sub-directory)? Otherwise the old/wrong preprocessed images and features were actually re-used.

If you did, I can only assume that your machine ran out of memory. Image processing is a memory-hungry task, which requires large amounts of RAM (sometimes > 32 GB). Maybe you should monitor your memory consumption during training. If you run out of memory again, there is little what we can do -- except for reducing the amount of training images in your database.

[daniel]

Yes, I did specify a new sub-directory. I am now trying to run the experiment with another algorithm (pca), what preprocessor and extractor should I use?

Also, at one point, I got this message on the terminal: bob.bio.base@2016-10-14 10:12:32,963 -- INFO: - Projection: loading training data, and after a while the process was killed again. Is there any way to increase verbosity so that it prints more information on the terminal, other than using the 3 '--verbose' levels that I used?

[Manuel Günther]

This looks very much like you are running out of memory. You should monitor your system memory using system tools like ``top``. As written above, depending on your configurations (e.g., the size of your preprocessed images, the number of files, the type of features), a lot of memory might be required; even 32 GB of RAM sometimes is not sufficient.

For PCA, usually the ``linearize`` extractor is used, which simply aligns all pixels into one vector (eigenfaces). Again, depending on the image resolution returned by your preprocessor, and on the number of training set images, some RAM might be required. We usually use an image resolution of 64x80 pixels.

The verbosity is limited to level 3. Which kind of information are written on the console depends on the programmer of the tools. In most cases, no DEBUG information (level 3) is printed, but only INFO (level 2) information. 

Anyways, I don't think that any of the files is problematic -- you would have received a different error message.

I hope this helps

Manuel

[daniel]

Manuel,

You are right, it is a memory problem. I was able to run the verify.py and evaluate.py and generated the ROC curve, but for a small part of the database (35 people, out of over 700). I was wondering if there is anything i can do lower the memory required, since you said 32GB of RAM might not even be enough and I only have 8GB.

Thanks again for taking the time to help me, I couldn't do it without your help.
Daniel

[Manuel Günther]

You are welcome.

High amounts of memory are only required during training. Hence, when reducing the training set ("world") size, you should be able to run the program with smaller amounts of memory. The rest of the processing (preprocessing, feature extraction, projection, enrollment and scoring) does not require that much memory 

Another option would be to reduce the image resolution of the preprocessed images, i.e., in order to have shorter feature vectors. However, 8GB is really not a lot for image processing tasks -- and I guess that half of the memory is already taken by the operating system. You might want to increase the swap space, but in that case, it will slow down processing and the rest of your machine, so this is not recommended.

Maybe, we can also benefit from your experiences. I think that you have created a database interface for a dataset (color FERET) that we don't provide yet. In case you want to share the interface with us (and the rest of the world), you are welcome to do so. 

One way to do so would be to create a (private) git repository on GitHub or bitbucket, and invite me or other Idiapers to your repo. User names will be provided on request.

Cheers

Manuel

[daniel]

Hi Manuel,

sorry it took me so long to respond. I have created a repository on bitbucket, and if you want to give me your username I will invite you to it.

Thanks

[Manuel Günther]

Hi Daniel,

we had a paper deadline, sorry for replying late.

My bitbucket username is 'mgunther'. I would be hapy to review your code and -- in case -- include it into our framework. 

@Amir: I might need some help with that -- as I am not allowed to create new repositories in Idiap's GitLab anymore. I will contact you personally regarding to that.

Cheers

Manuel

[Manuel Günther]

Dear Daniel,

I was expecting to have an implementation of the original FERET protocol, i.e., using DUP I, DUP II, fafb and fafc. However, I only find a single protocol in you code. 

As I do not possess the colorFERET database, I would be interested whether these protocols are still valid in colorFERET? 

- If yes, in order to be comparable with previously reported results on this database, we would need to implement those default protocols, and your implementation would unfortunately be useless for us. 

- If no, could you please tell me, where you found the protocol that you have implemented? I have searched online for an (updated) colorFERET protocol, but I couldn't find anything newer than the DUP I, DUP II, fafb and fafc protocols.

@idiapers: Do you by any chance obtained a copy of the colorFERET database? Can you help us out here?

Cheers

Manuel

[Tiago Freitas Pereira]

Hi Manuel,

We don't have the colorFeret dataset here.

I will open a ticket so the help desk can download it.

Cheers

--
-- You received this message because you are subscribed to the Google Groups bob-devel group. To post to this group, send email to bob-...@googlegroups.com. To unsubscribe from this group, send email to bob-devel+unsubscribe@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/bob-devel or directly the project website at http://idiap.github.com/bob/
---
You received this message because you are subscribed to the Google Groups "bob-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bob-devel+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Tiago

[Pavel Korshunov]

Hi guys,

I actually have full FERET database downloaded, so I can pass it to the system guys.

-pavel

Dr. Pavel Korshunov

Biometric group

Idiap Research Institute

Rue Marconi 19

CH - 1920 Martigny

Switzerland

Room: 207

Tel:  +41 27 721 77 56

Fax: +41 27 721 77 12

[Tiago Freitas Pereira]

Pavel, you cannot do that, Idiap has to sign an agreement in order to be able to download the data.

[Pavel Korshunov]

Ah, you are right.