setting up conda env in google colab?

[S OB]

Hello Rowan, 

We are a team of students working with the r2c model and we've opted to use google drive and colab to work on a shared version of the code. 

We've uploaded the git repo, and the embeddings, annotations etc of the model and dataset to google drive fine, but struggling to get colab to cooperate when it comes to installing conda and activating the r2c env. 

I've tried using both chunks of shell commands in the r2c/README, and I've tried using miniconda as per this blog. But with either method running:

!source activate r2c OR !source activate r2c && export PYTHONPATH=/usr/local/envs/r2c

runs without visibly complaining, but if I check whether it actually worked using:

import sys

sys.prefix == sys.base_prefix

then it doesn't actually seem as though the virtual env is running. Furthermore if I try to run:

!python train.py -params multiatt/default.json -folder saves/flagship_answer

I get a "No module named 'numpy'" error - which I assume is because the environment is definitely not running. 

Any ideas as to why this isn't working and how to get around it? 

Thanks :) 

[Rowan Zellers]

hey! that sounds frustrating. Unfortunately I don't think I can be much help here because I've never used Google Colab before :'( . However, I don't think you actually should need conda. There might be a better way to get the pytorch that I used back in 2018 when I was working on this initially :)

that said... these days if you can it might be better to use more modern tools, like I've heard great things about Huggingface Transformers for instance, and Pytorch has also gotten a lot better over the last few years :) though I mostly use tensorflow these days myself, it's got somewhat of a learning curve...

thanks,

Rowan

[S OB]

Hi, 

I didn't realise you could use something other than conda :) thanks for pointing that out. And after a little more research, it looks like it's not really ideal to use conda in google colab because "the only option is to create the environment each time, and run all your code in one cell" - from this post.

Thanks for suggesting the huggingface transformer library, I'd never heard of it but will definitely check it out because it also supports GPT-2 which is something we're trying to experiment with too! 

Many thanks!