Using bob.bio.base for different scenarios

[Tim Morawietz]

Hello,

I am trying to set up Bob.Bio.base/.face/.video for different scenarios. One would be running verifications and identifications from file system without the need of databases. So I would like to use verify.py the following way: verify.py /home/myimg/img1.jpg /home/myimg/img2.jpg …  and identification like this: images on file system in myimg

|--myimg
  |--p1
    -img1.jpg
    -img2.jpg
    …
  |--p2
    …

Verify.py /home/testface.jpg /home/myimg/ …

Furthermore I want to use bob for open set real-time identification with webcam enrollment + simple/basic GUI (Demo e-gate system if possible with PAD  features). Does bob.bio.base + video + face (Bobs example installation) support those scenarios. I know bob is based around reproducible research but can it also be used for custom applications or integrated in them?

How could I implement new features in bob like webcam enrollment with GUI, if it does not support it? How well does one need to know you bob system for such plans? Note that I don’t know Python at all, would learn it though.

 

Regards,
Tim

[Amir Mohammadi]

Hi Tim,

If you search through this mailing list, you will find similar questions to your question.

Basically, bob.bio.base comes with other scripts than verify.py(like enroll.py) which are not documented properly.

For doing open set real-time identification with webcam enrollment, you need to either use those scripts if they are suitable or write your own scripts.

Best,

Amir

--
-- You received this message because you are subscribed to the Google Groups bob-devel group. To post to this group, send email to bob-...@googlegroups.com. To unsubscribe from this group, send email to bob-devel+...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/bob-devel or directly the project website at http://idiap.github.com/bob/
---
You received this message because you are subscribed to the Google Groups "bob-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bob-devel+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Manuel Günther]

Dear Tim,

a reasonably simple way to run your first experiment (with files in directories) would be to create a filelist database for your files: https://www.idiap.ch/software/bob/docs/bob/bob.bio.base/stable/filelist-guide.html

Using this filelist database, you can plug in your images into the biometric recognition toolchain defined in bob.bio.base, i.e., by selecting a proper preprocessor, extractor and algorithm: https://www.idiap.ch/software/bob/docs/bob/bob.bio.base/stable/index.html

While the bob.bio packages might not be the best solution for a webcam-based GUI, bob per se can be useful. We once had a stand-alone face recognition example for three different face recognition systems using only core bob functionality (and no bob.bio): https://gitlab.idiap.ch/bob/bob.example.faceverify 

While I am not sure that this package still is working, you might get some insight on how to implement your own face recognition system based on bob. 

A webcam interface, however, is not part of bob and needs to be implemented using other python libraries, OpenCV for example.

Please be aware that the image format between OpenCV and bob is different, though, so you have to convert between these two formats.

Best regards

Manuel

[Tim Morawietz]

Thanks Amir and Manuel,

I’m currently looking into that.
I just can’t figure out how to create my dev/for_models.lst writing them by hand can’t be the solution. Do you have a script for that? I have not found one. 

Regards,
Tim

[Amir Mohammadi]

Hi Tim,

You have to create them by hand or create using your own scripts since it's specific to your dataset.

Best,
Amir

--

[Tim Morawietz]

Hi Amir,

I have created my file lists and my db configuration. In dev/  for_models.lst and for_probes.lst in eval/ for_models.lst and for_probes.lst

I get the following output if i try to run verify.py:

$ verify.py --database/home/tim/protocoltest/testdb.py -p base -e eigenface -a pca --sub-directory testexperi -vv

bob.bio.base@2017-11-14 02:14:48,262 -- ERROR: During the execution, an exception was raised: Invalid group 'world'. Valid values are ['dev', 'eval'], or lists/tuples of those

Traceback (most recent call last):

  File "/home/tim/miniconda3/bin/verify.py", line 6, in <module>

    sys.exit(bob.bio.base.script.verify.main())

  File "/home/tim/miniconda3/lib/python3.6/site-packages/bob/bio/base/script/verify.py", line 435, in main

    verify(args, command_line_parameters)

  File "/home/tim/miniconda3/lib/python3.6/site-packages/bob/bio/base/script/verify.py", line 415, in verify

    if not execute(args):

  File "/home/tim/miniconda3/lib/python3.6/site-packages/bob/bio/base/script/verify.py", line 262, in execute

    indices = tools.indices(fs.original_data_list(groups=tools.groups(args)), None if args.grid is None else args.grid.number_of_preprocessing_jobs),

  File "/home/tim/miniconda3/lib/python3.6/site-packages/bob/bio/base/tools/FileSelector.py", line 113, in original_data_list

    return self.database.all_files(groups=groups,add_zt_files=self.zt_norm)

  File "/home/tim/miniconda3/lib/python3.6/site-packages/bob/bio/base/database/filelist/query.py", line 223, in all_files

    files = self.objects(groups, self.protocol, **self.all_files_options)

  File "/home/tim/miniconda3/lib/python3.6/site-packages/bob/bio/base/database/filelist/query.py", line 623, in objects

    default_parameters=self.groups(protocol, add_subworld=False))

  File "/home/tim/miniconda3/lib/python3.6/site-packages/bob/db/base/database.py", line 109, in check_parameters_for_validity

    default_parameters)

  File "/home/tim/miniconda3/lib/python3.6/site-packages/bob/db/base/utils.py", line 356, in check_parameters_for_validity

    (parameter_description, parameter, valid_parameters))

ValueError: Invalid group 'world'. Valid values are ['dev', 'eval'], or lists/tuples of those

I tried to add --groups [dev,eval] and it changed to "Invalid group '[dev,eval]' " instead of world.

My guess is that i failed to set up my configuration. I get confused by bobs documentation. 

https://www.idiap.ch/software/bob/docs/bob/bob.bio.base/stable/implementation.html#defining-your-own-database

https://gitlab.idiap.ch/bob/bob.bio.spear/blob/master/bob/bio/spear/config/database/timit.py

What i can find, following the 2nd link looks different from what is described by the 3rd link source.  

https://www.idiap.ch/software/bob/docs/bob/bob.bio.base/stable/filelist-guide.html#protocols-and-file-lists

If i use db = ... as described following link3  instead of database: "The desired keyword 'database' dose not exist in any of you configuration files : /home/tim/protocoltest/testdb.py"

Maybe i am missing something obvious. 

Training any algorithms is not planed for now. 

I just want to run the baseline experiments from bob documentation on a custom file based db  (100 people from FERET)
 

My current conifg (probably causing my problem): 

#!/usr/bin/env python

# import pkg_resources

import bob.bio.base

from bob.bio.face.database import FaceBioFile

#TESTDB_DIR = "home/tim/protocoltest/testdb/"

# ,bio_file_class=FaceBioFile,original_extension=".jpg"

database = bob.bio.base.database.FileListBioDatabase('testdb/','testdb')

database.objects()

I hope that's  a sufficient amount of information. 

Regards,

Tim

[Amir Mohammadi]

Hi Tim,

You should tend to create a configuration like timit.

I understand that the documentation for file-list databases are outdated.

We have an issue for that too: https://gitlab.idiap.ch/bob/bob.bio.base/issues/58

If you feel like you can help, at the end of this discussion, you are encouraged to improve

the documentation by opening a pull request and changing the guide here (on our mirror):
https://github.com/bioidiap/bob.bio.base/blob/master/doc/filelist-guide.rst

Generally, anywhere you find confusing, please try to improve after you find out the solution.

We always appreciate feedback from new comers.

Now back to your question,

I think the problem you have now is that the algorithm and the extractor you have chosen require training.

So database.objects(groups='world') will be called by verify.py internals anyway to train those blocks.

So you need some training data to use those extractor and algorithms since they do not come pre-trained.

Another option would be to train those blocks with another database. This needs a separate call

to verify.py using a different database in configuration.

Some details:

1- you do not need to call database.objects() in your configuration file it will be called by verify.py

2- You need to provide the bio_file_class as you have commented out.

3- you can call verify.py like verify.py --groups dev eval to mention which groups you want.

4- It is better to put everything in a single config file as explained here:

https://www.idiap.ch/software/bob/docs/bob/bob.bio.base/stable/experiments.html#running-experiments-part-i

5- Here is an example to create a file list based database:

#!/usr/bin/env python
import bob.bio.base

# if your database is made of face images
from bob.bio.face.database import FaceBioFile

# if your database is made of video files
from bob.bio.video.database import VideoBioFile

# if your database is made of audio files (in case of speaker recognition)
from bob.bio.spear.database import AudioBioFile

# if your database is made of finger images
from bob.bio.finger.database import VeinBioFile

# here I assume you have images not video files:
database = bob.bio.base.database.FileListBioDatabase(
    '/path/to/filelists/', # path to the folder which has the dev, eval folders which in turn have the file lists.
    name='testdb', # name your database here. Do not use spaces in the name.
    bio_file_class=FaceBioFile, # this is important. You need to use the BioFile class which is compatible with your actual files
    original_directory='/path/to/actual_images',
    original_extension=".jpg")

Please let me know if it worked or you have more questions.

Thanks,

Amir

--

[Tim Morawietz]

Hi Amir,

Sadly it dose not work jet. I have created a bare minimum setup for some try and error testing with only 2 images.

But the new problem: file '...test.jpg' is not readable dose not make sense to me. I checked permissions and even did what should never be done umask 000. My images can opened by other programs without anomalies.

verify.py can find my file-lists and images but somehow not open them.

verify.py testdb.py --groups dev eval
bob.bio.base@2017-11-16 03:54:02,657 -- INFO: - Preprocessing: processing 2 data files from directory 'home/tim/' to directory 'temp/testexperi/preprocessed'

bob.bio.base@2017-11-16 03:54:02,658 -- ERROR: During the execution, an exception was raised: File - constructor: C++ exception caught: 'file 'home/tim/home/tim/00002_930831_fa.jpg' is not readable'

Traceback (most recent call last):

  File "/home/tim/miniconda3/bin/verify.py", line 6, in <module>

    sys.exit(bob.bio.base.script.verify.main())

  File "/home/tim/miniconda3/lib/python3.6/site-packages/bob/bio/base/script/verify.py", line 435, in main

    verify(args, command_line_parameters)

  File "/home/tim/miniconda3/lib/python3.6/site-packages/bob/bio/base/script/verify.py", line 415, in verify

    if not execute(args):

  File "/home/tim/miniconda3/lib/python3.6/site-packages/bob/bio/base/script/verify.py", line 264, in execute

    force = args.force)

  File "/home/tim/miniconda3/lib/python3.6/site-packages/bob/bio/base/tools/preprocessor.py", line 71, in preprocess

    data = preprocessor.read_original_data(file_object, original_directory, original_extension)

  File "/home/tim/miniconda3/lib/python3.6/site-packages/bob/bio/base/utils/io.py", line 85, in read_original_data

    return biofile.load(directory, extension)

  File "/home/tim/miniconda3/lib/python3.6/site-packages/bob/db/base/file.py", line 132, in load

    return bob.io.base.load(path)

  File "/home/tim/miniconda3/lib/python3.6/site-packages/bob/io/base/__init__.py", line 149, in load

    return File(inputs, 'r').read()

RuntimeError: File - constructor: C++ exception caught: 'file 'home/tim/home/tim/00002_930831_fa.jpg' is not readable'

I tried changing my original_directory so the file path given by this error log would look correct. 

# original_directory or original_directory='' results in  home/tim/00002_930831_fa.jpg where it is. 

The first home/tim is part of original_directory the rest home/tim/00002_930831_fa.jpg has to be out of my file-list e.g for_models.lst

I still could try different image extensions or reduce there size. Currently 512x768 40kB .jpg

I attempted to do some debugging but verify.py with its imports and dependencies can't just be loaded "as is"  that easily.

      

CptureFlist.JPG shows my setup. 

Regards,

Tim 

[Amir Mohammadi]

Hi Tim,

One way to debug the code is to enter a debugger as soon as it fails.

To do this, you can use ipython:

Instead of :

verify.py testdb.py --groups dev eval

you can run:

ipython --pdb -- `which verify.py` testdb.py --groups dev eval

Make sure ipython is installed in your bob conda environment though.

try "which ipython" to see if it is coming from your bob environment.

Also, could you attach the image that is failing?

Thanks,

Amir

[Tim Morawietz]

Hi Amir,

I use images from FERET and tried my own.

Regards,

Tim  

[Amir Mohammadi]

Do you get this error with any image ?

--

[Tim Morawietz]

Did some tests all images from FERET fail and my own captured with my Smartphone in hd.

Added debugger log.

[Amir Mohammadi]

I think your path to the file is wrong.

The error is bad. It says it is not readable but actually it should say the file does not exist.

Best,

Amir

On Thu, Nov 16, 2017 at 2:49 PM Tim Morawietz <timst...@gmail.com> wrote:

Did some tests all images from FERET fail and my own captured with my Smartphone in hd.

Added debugger log.

--

[Tim Morawietz]

Hi Amir,

Thanks for all your help.My file-list DBs are now set up properly and i can run experiments.

I don't know if that is how it should be but it works for me:

import bob.bio.base

from bob.bio.face.database import FaceBioFile

database = bob.bio.base.database.FileListBioDatabase('testdb',  #relative to my DBconfig.py dir 'testdb' same place as testdb.py

name = 'testdb',

bio_file_class = FaceBioFile,

protocol='p1',

original_directory = "/home/tim", #img dir starting at root "/home/tim/imgdir"--> files can't be found  

original_extension = ".jpg",

database = 'testdb')

preprocessor = ''  #base , face-detect 

extractor = ''     #linearize , eigenface

algorithm = ''     #gabor-jet , distance-cosine

sub_directory = 'testexperi'

verbose = 2

My file lists contain absolute paths like /home/tim/imgdir/001/img1

I still don't know why my original_directory dose not accept the full path, only to its parent dir. 

Regards,

Tim

[Amir Mohammadi]

Hi Tim,

You seem to be confused about how filelists should look like.

I will try to explain. Assume you have two images with absolute paths like this:

/home/tim/my_image_dataset/train_images/identity_1/image_1.jpg
/home/tim/my_image_dataset/train_images/identity_2/image_1.jpg

In your .lst files, you will refer to them as:

train_images/identity_1/image_1.jpg
train_images/identity_2/image_1.jpg

Notice that the paths do not start with "/" above. For example "/train_images/identity_2/image_1.jpg" is wrong.

And when you create the configuration in Python, you do like this:

database = FileListBioDatabase( .... original_directory='/home/tim/my_image_dataset/')

I hope that clears it.

Best,

Amir

[Manuel Günther]

Dear Tim,

as Amir has pointed out, you should have relative paths inside your file list database. While you will be able to use the filelist database even with absolute paths, the bob.bio framework might do weired things when paths are absolute.

What Amir has missed is that you should not store the extension of your file names inside your database, but rather specify them as the 'original_extension' (as you already do). Hence, your file lists should rather contain files like:

train_images/identity_1/image_1

train_images/identity_2/image_1

Please note that different types of file lists (training list, model list, scoring list/probe list) will have different formats and kinds of information that need to be stored, please refer to the documentation for the different formats: https://www.idiap.ch/software/bob/docs/bob/bob.bio.base/stable/filelist-guide.html#creating-file-lists

@bob: I am not sure if we currently test that file names in list files are relative. Maybe we should add a test for that before blindly using the paths.

Manuel