bob.measure negative threshold

[Tim Morawietz]

Hi 

I encountered  a solved bug Issue #35.

My bob.measure ver 2.4.1

My bob.bio.base ver 3.2.1 

Python 3.6.2 

Is the fix deployed for all or only internal? 

What code adjustments need to be done?

Regards,

Tim

[Amir Mohammadi]

Hi,

A version of bob.measure is not released with that fix yet.

This is the merge request that fixed that bug:
https://gitlab.idiap.ch/bob/bob.measure/merge_requests/42/diffs

Best,

Amir

--
-- You received this message because you are subscribed to the Google Groups bob-devel group. To post to this group, send email to bob-...@googlegroups.com. To unsubscribe from this group, send email to bob-devel+...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/bob-devel or directly the project website at http://idiap.github.com/bob/
---
You received this message because you are subscribed to the Google Groups "bob-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bob-devel+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Tim Morawietz]

Hi Amir,

I have added the changes but it dose not work for me.

My test_scripts.py appears to be identical to this one. 

...
def test_apply_threshold():

  # sanity checks

  assert os.path.exists(TEST_SCORES)

  cmdline = [

      '0.5',

      TEST_SCORES,

  ]

  from .script.apply_threshold import main

  nose.tools.eq_(main(cmdline), 0)

  cmdline = [

      '--',

      '-0.5',

      TEST_SCORES,

  ]

  nose.tools.eq_(main(cmdline), 0)

def test_compute_perf_5col():
...

Do i need to update something? It should recognizance the changes after editing by itself right?  

Regards,

Tim

[Amir Mohammadi]

Hi Tim,

It is not a good idea to edit the files that are installed by conda since they are hard-linked to your conda cache.

This will corrupt your cache.

You can however install bob.measure from source checkout:

$ git clone https://gitlab.idiap.ch/bob/bob.measure.git

$ cd bob.measure

# activate bob's conda environment.

# Something like "source activate bob_py3". Depends on what you have installed

$ source activate bob_py3 # the name might be different on your computer

$ conda install bob-devel=2017.11.28

# activate it again. this is needed since you just installed bob-devel

$ source deactivate

$ source activate bob_py3 # the name might be different on your computer

# run buildout

$ buildout

# run buildout's python

$ bin/python

>>> import bob.measure

>>> bob.measure

<module 'bob.measure' from '.../bob.measure/bob/measure/__init__.pyc'> # this should point to your local bob.measure installation

You would have all the required scripts in your local bin folder:

$ ls bin

... bob_apply_threshold.py ...

This is partly explained in detail here:
https://www.idiap.ch/software/bob/docs/bob/bob.extension/stable/pure_python.html#building-your-package

Best,

Amir

--

[Tim Morawietz]

Hi Amir,

It seems that i'm a magnet with only one purpose, attracting all bugs and anomalies.

conda-env list showed me two environments bob_py3(...miniconda3/envs/bob_py3) and root(* /home/tim/miniconda3).

I can't complete buildout under env bob_py3, adjusting the buildout.cfg dose not help. (Setting up index/find-links =  https://pypi.python.org/simple/ ).

I checked if can access the needed files under root and it can but throws other exceptions:

(bob_py3) tim@ubuntu:~/bob.measure$ buildout

Download error on https://pypi.python.org/simple/bob.buildout/: [Errno 110] Connection timed out -- Some packages may not be found!

Couldn't find index page for 'bob.buildout' (maybe misspelled?)

Download error on https://pypi.python.org/simple/: [Errno 110] Connection timed out -- Some packages may not be found!

Getting distribution for 'bob.buildout'.

Couldn't find index page for 'bob.buildout' (maybe misspelled?)

While:

  Installing.

  Loading extensions.

  Getting distribution for 'bob.buildout'.

Error: Couldn't find a distribution for 'bob.buildout'.

(root) tim@ubuntu:~/bob.measure$ buildout

Creating directory '/home/tim/bob.measure/bin'.

Creating directory '/home/tim/bob.measure/parts'.

Creating directory '/home/tim/bob.measure/develop-eggs'.

Develop: '/home/tim/bob.measure/.'

bob.buildout.tools: taking requirement `bob.extension' (2.4.5) from `/home/tim/miniconda3/lib/python3.6/site-packages'

bob.buildout.tools: taking requirement `bob.blitz' (2.0.14) from `/home/tim/miniconda3/lib/python3.6/site-packages'

bob.buildout.tools: taking requirement `bob.core' (2.1.9) from `/home/tim/miniconda3/lib/python3.6/site-packages'

bob.buildout.tools: taking requirement `bob.math' (3.0.0) from `/home/tim/miniconda3/lib/python3.6/site-packages'

bob.buildout.tools: taking requirement `bob.io.base' (3.0.2) from `/home/tim/miniconda3/lib/python3.6/site-packages'

bob.buildout.tools: taking requirement `matplotlib' (2.0.2) from `/home/tim/miniconda3/lib/python3.6/site-packages'

bob.buildout.tools: taking requirement `docopt' (0.6.2) from `/home/tim/miniconda3/lib/python3.6/site-packages'

running develop

running egg_info

creating bob.measure.egg-info

writing bob.measure.egg-info/PKG-INFO

writing dependency_links to bob.measure.egg-info/dependency_links.txt

writing entry points to bob.measure.egg-info/entry_points.txt

writing requirements to bob.measure.egg-info/requires.txt

writing top-level names to bob.measure.egg-info/top_level.txt

writing manifest file 'bob.measure.egg-info/SOURCES.txt'

reading manifest file 'bob.measure.egg-info/SOURCES.txt'

reading manifest template 'MANIFEST.in'

writing manifest file 'bob.measure.egg-info/SOURCES.txt'

running build_ext

building 'bob.measure.version' extension

creating build

creating build/temp.linux-x86_64-3.6

creating build/temp.linux-x86_64-3.6/bob

creating build/temp.linux-x86_64-3.6/bob/measure

gcc -pthread -B /home/tim/miniconda3/compiler_compat -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Wno-strict-aliasing -DBOB_EXT_MODULE_PREFIX="bob.measure" -DBOB_EXT_MODULE_NAME="version" -DBOB_EXT_ENTRY_NAME=PyInit_version -DBOB_EXT_MODULE_VERSION="2.5.0b0" -DHAVE_BOOST=1 -DBOOST_VERSION="1.61.0" -DHAVE_HDF5=1 -DHAVE_BLITZ=1 -DBLITZ_VERSION="0.10" -DPY_ARRAY_UNIQUE_SYMBOL=BOB_NUMPY_C_API -DNO_IMPORT_ARRAY=1 -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -I/home/tim/miniconda3/lib/python3.6/site-packages/bob/blitz/include -I/home/tim/miniconda3/lib/python3.6/site-packages/bob/extension/include -I/home/tim/miniconda3/lib/python3.6/site-packages/bob/core/include -I/home/tim/miniconda3/lib/python3.6/site-packages/bob/io/base/include -I/home/tim/miniconda3/include -I/home/tim/miniconda3/lib/python3.6/site-packages/bob/math/include -I/home/tim/miniconda3/include/python3.6m -c bob/measure/version.cpp -o build/temp.linux-x86_64-3.6/bob/measure/version.o -std=c++0x -pthread -isystem /home/tim/miniconda3/lib/python3.6/site-packages/numpy/core/include -isystem /home/tim/miniconda3/include

creating build/lib.linux-x86_64-3.6

creating build/lib.linux-x86_64-3.6/bob

creating build/lib.linux-x86_64-3.6/bob/measure

g++ -pthread -shared -B /home/tim/miniconda3/compiler_compat -L/home/tim/miniconda3/lib -Wl,-rpath=/home/tim/miniconda3/lib,--no-as-needed build/temp.linux-x86_64-3.6/bob/measure/version.o -L/home/tim/miniconda3/lib -L/home/tim/miniconda3/lib/python3.6/site-packages/bob/core -L/home/tim/miniconda3/lib/python3.6/site-packages/bob/io/base -L/home/tim/miniconda3/lib/python3.6/site-packages/bob/math -L/home/tim/miniconda3/lib -R/home/tim/miniconda3/lib -lbob_core -lbob_io_base -lbob_math -l:libboost_system.so.1.61.0 -lblitz -lpython3.6m -o build/lib.linux-x86_64-3.6/bob/measure/version.cpython-36m-x86_64-linux-gnu.so

g++: error: unrecognized command line option ‘-R’; did you mean ‘-R’?

error: command 'g++' failed with exit status 1

While:

  Installing.

  Processing develop directory '/home/tim/bob.measure/.'.

An internal error occurred due to a bug in either zc.buildout or in a

recipe being used:

Traceback (most recent call last):

  File "/home/tim/miniconda3/lib/python3.6/site-packages/zc/buildout/buildout.py", line 2123, in main

    getattr(buildout, command)(args)

  File "/home/tim/miniconda3/lib/python3.6/site-packages/zc/buildout/buildout.py", line 658, in install

    installed_develop_eggs = self._develop()

  File "/home/tim/miniconda3/lib/python3.6/site-packages/zc/buildout/buildout.py", line 901, in _develop

    zc.buildout.easy_install.develop(setup, dest)

  File "/home/tim/miniconda3/lib/python3.6/site-packages/bob/buildout/extension.py", line 213, in develop

    zc.buildout.easy_install.call_subprocess(args)

  File "/home/tim/miniconda3/lib/python3.6/site-packages/zc/buildout/easy_install.py", line 165, in call_subprocess

    % repr(args)[1:-1])

Exception: Failed to run command:

'/home/tim/miniconda3/bin/python', '/tmp/tmp3quqf3nz', '-v', 'develop', '-mxN', '-d', '/home/tim/bob.measure/develop-eggs/tmpsg16eejdbuild'

I have never used conda before but it just can't be that restrictive with editing installed files. I don't get why changing some lines in a hard linked file causes issues. I can change hard liked files from other projects (not using conda) just fine and immediately see different results (e.g. different exceptions, or not working at all).

Thanks for all your help,

Tim

[Amir Mohammadi]

Hi Tim,

Here is a relevant discussion on why it is not a good idea to modify the files of conda either by hand or by pip:
https://groups.google.com/a/continuum.io/forum/#!msg/conda/z9sor6HyWkI/PdVT7x8TAwAJ

Anyways, we have a guide now (written yesterday inspired from your question) for developing existing packages:
https://www.idiap.ch/software/bob/docs/bob/bob.extension/master/development.html

Could you give the guide a try, please? Try to avoid the TLDR section in my opinion (or read it first and at the end twice) and read the detailed guide.

Try to install bob-devel successfully in your conda environments first (or just create a new one as the guide suggests).

If you don't install bob-devel and don't activate the environment properly, our C++ code will not compile most likely.

to make sure you have bob-devel, do:

conda install bob-devel=2017.12.06

Thanks,

Amir

[Tim Morawietz]

Hi Amir,

I have managed to install a package in development mode. Could not have done it without the new guide.

Additional things i experienced:

I could not install bob-devel in one of my existing environments (root  or bob_py3) - reason different lib/package versions

(bob_py3) tim@ubuntu:~/bobmeasure/bob.measure$ conda install bob-devel=2017.12.06

Fetching package metadata .............

Solving package specifications: .

UnsatisfiableError: The following specifications were found to be in conflict:

  - bob-devel 2017.12.06* -> cmake 3.6.3 0 -> ncurses 5.9*

  - libedit

Use "conda info <package>" to see the dependencies for each package.

So I created a new env, as described in the guide but had to deal with a still existing buildout problem:

Download error on https://pypi.python.org/simple/bob.buildout/: [Errno 110] Connection timed out -- Some packages may not be found!

Couldn't find index page for 'bob.buildout' (maybe misspelled?)

Download error on https://pypi.python.org/simple/: [Errno 110] Connection timed out -- Some packages may not be found!

Getting distribution for 'bob.buildout'.

Couldn't find index page for 'bob.buildout' (maybe misspelled?)

While:

  Installing.

  Loading extensions.

  Getting distribution for 'bob.buildout'.

Error: Couldn't find a distribution for 'bob.buildout'.

Somehow if i try to run buildout in a new env it can't access bob.buildout. I fixed it by simply installing it manually into my new env with (new_env) user@ubuntu: conda install bob.buildout.   

The difference between the new guide and other documented parts(e.g. bob.bio.base [file-lists and databases] ) is like day and night. Guides like that are needed so people can get into bob without investing several weeks in source code reading so they understand how to do xyz. Definitely worth the effort writing a more in depth guide. Now i know how bob is put together and how / why scrips work the way they do (environment dependencies).

Best,

Tim  

[Amir Mohammadi]

Hi Tim,

Thank you for your feedback. I have added bob.buildout to the list of packages that get installed alongside bob-devel to avoid such issues:
https://gitlab.idiap.ch/bob/bob.conda/merge_requests/325

The documentation is written mostly by us and sometimes it is difficult to identify issues in the documentation since we know the underlying source code.

That is why we ask new-comers to raise issues about the documentation and also submit patches if possible.

Best,

Amir

--