Performance issiues using vanilla-biometrics

Skip to first unread message

Feb 19, 2021, 6:37:19 AM2/19/21
to bob-devel
Hi again,
I am aware that the package is not fully developed; I only want to let you know that there are some performance issues.
Reproduce UTFVP baseline experiments;
Installation method:
built locally, using commands:
conda create -n bdt -c bob bob.devtools
[activate bdt]
cd ./
bdt create -vv VEINS_DEV
[activate VEINS_DEV]
Indicated version in the ``version.txt``: 1.1.7b0
Experiments for the NOM dataset took some 15 hours for each -- dev and eval subset, using the command:
% ./bin/bob bio pipelines vanilla-biometrics nom mc utfvp -g eval -l local-parallel -o NOM_EVAL -vv
There were plenty of error messages during the execution (about delayed process execution for a specific core), but eventually, experiments finished producing the combined score file.
However, for ALL protocol:
% ./bin/bob bio pipelines vanilla-biometrics full utfvp mc -l local-parallel -o ALL -vv
 I had to stop execution after three days; eventually, the terminal was paralysed with messages like this:
distributed.worker - WARNING - Memory use is high but worker has no data to store to disk.  Perhaps some other process is leaking memory?  Process memory: 47.60 GB -- Worker memory limit: 67.36 GB

Hope this helps. Will proceed using the stable version. One of the main misunderstandings arises from the fact that if one google e.g., more often than not the master branch version of the docs is the first result, instead of the stable version.

Hope this helps,

Some screenshots:


Tiago Freitas Pereira

Feb 19, 2021, 7:12:18 AM2/19/21
Hi Teodors,

Glad to hear that there's something testing `` :-)
Let me try to understand your issue. I'm not familiarized with the algorithms and database protocols of

So, usually, how long does it take to run the experiment (let's say the `nom` protocol)?

Second, to avoid the memory increase, add the option `--checkpoint` in your command.
Otherwise, all your experiment will be kept in memory until the score writing.

Let me know how it goes



-- You received this message because you are subscribed to the Google Groups bob-devel group. To post to this group, send email to To unsubscribe from this group, send email to For more options, visit this group at or directly the project website at
You received this message because you are subscribed to the Google Groups "bob-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
To view this discussion on the web, visit


Tiago Freitas Pereira

Feb 19, 2021, 7:18:48 AM2/19/21
Hi Teodors,

I'm requesting access to the UTFVP dataset, so I can test this with you.



Feb 19, 2021, 3:27:28 PM2/19/21
to bob-devel
Today, with the stable version,  nom protocol, both group dev and eval experiment execution time was about 2 hours; full took some four.
As I mentioned before, there were error messages constantly; but I didn't save them. If interest for you, can reproduce it; I don't have any clue about the fancy software version/implementation details.


Reply all
Reply to author
0 new messages