Issues when mixing conda and pip and uninstalling

[Manuel Günther]

I have found an issue when using a mixed environment, where I installed bob with conda and some other bob packages with pip. While this works fine during installation and usage, uninstalling the packages with pip makes the environment unusable. Here is a path to recreate this issue:

1. install bob with conda:

$ conda create -n bob_test --override-channels -c https://www.idiap.ch/software/bob/conda -c defaults python=2 bob

2. activate conda environment

$ conda activate bob_test

3. install bob.db.lfw via pip

(bob_test) $ pip install bob.db.lfw

4. test the installation:

(bob_test) $ python
>>> import bob.db.lfw, bob.io.image

5. uninstall bob.db.lfw with pip

(bob_test) $ pip uninstall bob.db.lfw

6. cannot use bob anymore

(bob_test) $ python
>>> import bob.io.image
No module named io.image

7. cannot install any bob package anymore

(bob_test) $ pip install bob.db.lfw
  Downloading bob.db.lfw
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-19sMQi/bob.db.lfw/setup.py", line 8, in <module>
        from bob.extension.utils import load_requirements, find_packages
    ImportError: No module named extension.utils

The reason is that the uninstallation also uninstalled the __init__.py files that declare the bob namespace. After copying back those files into conda/envs/bob_test/site_packages/bob (and .../bob/db in this case) the installation will be working again.

Since there is most probably nothing that we can do about this, I am not opening an issue somewhere. I think this is rather an issue of pip. I am mainly positing this solution in case someone has the same issue.

Cheers

Manuel

[Amir Mohammadi]

Manuel,

Thanks for pointing this out.

In general, using pip in a conda environemnt can lead to the corruption of all your environments. See: https://stackoverflow.com/questions/43879119/installing-tensorflow-in-cloned-conda-enviroment-breaks-conda-enviroment-it-was?noredirect=1#comment74854400_43879119

and https://groups.google.com/a/continuum.io/forum/#!topicsearchin/conda/Installing$20a$20package$20in$20a$20cloned$20/conda/z9sor6HyWkI

I would avoid using pip in all cases. We have also made it a requirement that all bob packages should have a conda package. If you find a public bob package with no conda package, that is a bug.

We also have our own conda channel in bob.conda where we package our missing dependencies. This is all done to avoid pip installations in all cases.

Now to fix your environment, you can use conda install --force but using it is not easy since it does not force install the dependencies. So to use it, you will have to run a command like:

conda install --force bob.extension bob.blitz bob.core bob.io.base ....

listing all installed bob packages which you can get by calling conda list:

conda list bob

Best,

Amir

--
-- You received this message because you are subscribed to the Google Groups bob-devel group. To post to this group, send email to bob-...@googlegroups.com. To unsubscribe from this group, send email to bob-devel+...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/bob-devel or directly the project website at http://idiap.github.com/bob/
---
You received this message because you are subscribed to the Google Groups "bob-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bob-devel+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Manuel Günther]

Amir,

thanks for adding this information.

I have tried to 

conda install --force bob

but this didn't do the trick -- I guess because all bob packages are just dependencies of the bob conda package.

Now I have copied the __init__.py files by hand, but I will try your solution the next time. Since only the __init__.py is missing, re-installing a single package should be sufficient to get that file back.

Cheers

Manuel