Tensorflow 2 installation in virtualenv

[Stefan Berg-Johansen]

If you have many Python projects running on your computer and would like to install the latest Tensorflow 2 without affecting any existing package dependencies, you may want to create a virtual environment for your Tensorflow-related work.

Here is how to do that on Linux.

Note: This is only a trick that may be helpful to some people, and is not at all essential to the course. If this looks obscure to you, don't worry, you probably don't need it.

Initially, make sure the basic python3, jupyter-notebook, python3-virtualenv and python3-pip (or equivalent) packages are already installed in the usual way on your system (i.e. via apt on Debian/Ubuntu).

Create a new environment and activate it:

~$ virtualenv -p python3 ./tfenv    # 'tfenv' is the (arbitrary) name of our new environment
~$ source ./tfenv/bin/activate
(tfenv) ~$                          # '(tfenv)' indicates new environment is active


Install tensorflow and ipython (and optionally any other packages you require) in the following way:

(tfenv) ~$ pip install tensorflow ipython


If pip complains about low disk space (e.g. due to small /tmp partition), just create a local tmp/ dir and tell pip to use that for the installation:

(tfenv) ~$ cd tfenv/
(tfenv) ~/tfenv$ mkdir tmp/
(tfenv) ~/tfenv$ TMPDIR=tmp/ pip install --cache-dir=tmp/ --build tmp/ tensorflow ipython

To connect the new environment with Jupyter, run

(tfenv) ~$ pip install ipykernel
(tfenv) ~$ python3 -m ipykernel install --user --name=tfenv

Now, start a Jupyter server (or restart it, if it was already running). In the "New Notebook" dropdown menu, the `tfenv` should appear. It lets you create an interactive jupyter notebook within the newly created environment.

To leave the virtual environment, simply close the shell or run

(tfenv) ~$ deactivate
~$

Hope this helps someone!

[Florian Marquardt]

Thank you very much for posting this!

[Stefan Berg-Johansen]

For analogous steps on Windows/Conda and macOS, see

• https://www.tensorflow.org/install/pip

• https://janakiev.com/blog/jupyter-virtual-envs/

[rober guerrero]

Hi, thanks for the tip. Firstly, sorry for my english and secondly, I've installed tensorfolw following your steps, but now when I run a code, my packages don't run such as numpy or matploit. I'd think that the env works for separating tensorflow of the rest of packages, what I mean is that inside tfenv works rest of packages+ tensorflow but outside tfenv only works the rest of packages. Do I install all the packages again inside tfenv or Can I made a link?

  

   

[Stefan Berg-Johansen]

Hi Rober, if you create the virtualenv with the `--system-site-packages` flag in the first step above, already installed system-wide packages will be available within the environment. In fact, this is what is suggested in the tensorflow installation guide. I personally don't like to do this and instead (re)install all required packages within the env. The price of this separation is a few 100 MBs of extra disk space, up to a few GBs maximum. It is a matter of preference.

Another detail is that the `--user` flag in the last step additionally creates a small folder in `.local/share/jupyter/kernels/` containing a JSON file letting jupyter know about your env. This lets you run different kernels with different environments, all out of the same jupyter server instance. I personally prefer this, but it means that you have to additionally remove that folder when you delete the environment (there is also an "uninstall" command that does this, found in the second linked article above).

So as you have already noticed, there are many configuration details. The main intention of this thread was simply to point out that virtualenvs exist and can be useful. The details can be investigated by web search, reading man pages and trying things out. Of course I'm happy to help, provided that I know the answer :)

Fortunately, it's quick and easy to remove environments (just delete the folder) and create new ones if you change your mind about something. Your actual notebooks, data, and results should be stored in a different location in any case.

[rober guerrero]

thanks you for your reply.

[Shilan]

Hi,

I want to install Tensorflow 2 as recommended for the course. But now I got a little worry because I have many python codes on my laptop relating to my studies.

Do you think installing TensorFlow will affect it? My system is macOS Sierra version 10.12.6.

Thanks in advance!

[Stefan Berg-Johansen]

Hi Shilan,

just know that `pip install` will upgrade existing packages in order to meet the dependencies of what you're installing. If you run `pip install tensorflow`, you may end up with newer versions of scipy, numpy, matplotlib et al. in your user/site installation.

In general that's fine, but if you want to guarantee non-interference with existing workflows, just install tensorflow in a virtualenv as detailed above.

`virtualenv` also seems to be the recommended installation procedure on tensorflow.org (although they don't state any reasons).

Best regards, Stefan

[Shilan]

Thanks for your quick response.