AWS AMIs for TensorFlow Magenta

[arsone]

Has anyone got a good machine image for AWS with all the necessary dependencies that they'd be willing to share? Having a TensorFlow Magenta AMI would go a long way towards lowering the barrier to entry for people. If I get a chance this week I'll test/create one and link to it here, but if anyone else has a good setup they want to share with the group feel free to post it. Thanks!

[David Plans]

My setup is OS X but I'm migrating to an Ubuntu build and I'd love to see an Ubuntu image (like planet CCRMA days) for this kind of work

On Tue, 14 Jun 2016 at 14:13, arsone <adsa...@gmail.com> wrote:

Has anyone got a good machine image for AWS with all the necessary dependencies that they'd be willing to share? Having a TensorFlow Magenta AMI would go a long way towards lowering the barrier to entry for people. If I get a chance this week I'll test/create one and link to it here, but if anyone else has a good setup they want to share with the group feel free to post it. Thanks!

--
Magenta project: magenta.tensorflow.org
To post to this group, send email to magenta...@tensorflow.org
To unsubscribe from this group, send email to magenta-discu...@tensorflow.org
---
You received this message because you are subscribed to the Google Groups "Magenta Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to magenta-discu...@tensorflow.org.

[Panagiotis Tigas]

+1

Or a docker image could be as helpful.

cheers,

p.

--

Panagiotis Tigas

http://ptigas.com/

@ptigas

[Brad Isenbek]

I just set up ami-e61dd98b - this should be launched as a GPU instance. Enjoy!

[Adam Salberg]

Beat me to it! I spent several hours today in CUDA versioning hell.

Currently running your ami on a g2.2xlarge instance, seems to be working great! The included bazel tests fail (I think they're hardcoded to look for libcudart.so.7.5 instead of 7.0? or maybe it's vice versa, can't remember), but everything else seems to work as promised. Thanks!

--

Magenta project: magenta.tensorflow.org
To post to this group, send email to magenta...@tensorflow.org
To unsubscribe from this group, send email to magenta-discu...@tensorflow.org
---
You received this message because you are subscribed to the Google Groups "Magenta Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to magenta-discu...@tensorflow.org.

--

914.261.8998 • adsa...@gmail.com • adamsalberg.com

[Brad Isenbek]

No problem! This is a warts-and-all build :) I spent a weekend going through the versioning minefield. Had to modify some of the TensorFlow source to accept the AWS GPU type. I've been running Magenta and some image recognition CNN's with no problems.

If you have not already - it's worth exploring spot instances for the g2's. I set my pricing to 'auto' and have had no issues with the instance being reclaimed. Create an AMI from time-to-time and you'll save big $$$.

[Blue Light]

Anyone make a more up-to-date Magenta AMI with GPU support? If not, can anyone provide some guidance about how the old one ami-e61dd98b may be updated to have the latest Magenta while keeping the GPU support? Or alternatively, just guidance for one to set-up an Magenta AWS instance with GPU support?

Thank you

[Anish Bhatnagar]

Just started from scratch on a p2.xlarge instance and saved down ami-2db72f4d, feel free to try it out. I couldn't find any p2 in N. California region so it's attached to Oregon.

If you want to DIY, I would suggest:

1. Start from this BitFusion image: https://github.com/bitfusionio/amis/wiki/Bitfusion-Boost-Ubuntu-14-Tensorflow-AMI

(the main advantage over starting from scratch is avoiding CuDDN installation which is slightly hairy)

2. Install general updates. sudo apt-get update

3. You probably want to install the latest version of cuda

wget https://developer.nvidia.com/compute/cuda/8.0/prod/local_installers/cuda-repo-ubuntu1604-8-0-local_8.0.44-1_amd64-deb

sudo dpkg -i cuda-repo-ubuntu1604-8-0-local_8.0.44-1_amd64-deb

4. Install conda and magenta by following the instructions in the magenta readme

5. After installing magenta and activating the environment,  replace the non-gpu tensorflow with a gpu-supported version:

pip install --ignore-installed --upgrade https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-1.1.0-cp27-none-linux_x86_64.whl

6. Do a reboot for good measure and you should be up and running.

--As a side note, it would be great if the magenta team came out with an official AMI for this or more simply a magenta package with gpu support.