Can Magenta be installed on BASH-ON-WINDOWS-LINUX-SUBSYSTEM ?

[Jeremy Ellis]

The anniversary edition of Windows 10 released  Aug 2, 2016  allows Ubuntu 14.04 to be run on windows!  This is a very big deal. 

I have got bash-on-windows working and am presently trying to install a few things that I am more familiar with than Tensorflow and that is running Android java phonegap installation scripts. I will eventually try to install Magenta but I already know that Bazel is going to be an issue. https://groups.google.com/a/tensorflow.org/d/msg/magenta-discuss/oXmgOUE996c/U2JoL9CeDQAJ

Hopefully all of this will work, but it is a very big step and will probably have lots of issues.

[Jeremy Ellis]

First couple of tricks.

When bash asks for a username you might want to type 

root

I used my name but Bazel makes a directory called 

~/.cache/bazel/_bazel_root

and then makes a link to a directory called

~/.cache/bazel/_bazel_jeremy

Another issue is many commands that don't need sudo on your own computer need sudo on windows. Might be easier just to type

sudo su

so every command is using sudo

My Bazel still does not work yet.

[Jeremy Ellis]

Quick update:

I have got java8 running and Bazel. Just have to remember what I did. Not sure how well they work but they do install!

[Steven Lamphear]

I've gotten everything installed, but haven't been able to get any Bazel commands to work. Way to go!

Are you using openjdk or Oracle's JDK? What output do you get for "java -version"?

-Steven

[Jeremy Ellis]

I have both java 7 (openjdk-7-jdk) and java 8 (jdk-8u5-linux-x64) installed and am switching back and forth while playing around with the android sdk and tensorflow bazel. My present issue is about whether to use 

sudo su

or whether to place a sudo in front of every command. Each situation has some positives. Bazel worked with sudo su .

Other things seem to work better in the virtual environment and some things don't like it.

Tensorflow works as does jupyter iPython notebook which can load in a windows web browser. That looks good for getting Tensorboard working.

My site is not anywhere near ready but if you want to have a look at it it is at 

https://github.com/hpssjellis/my-installs-for-bash-on-windows

[Jeremy Ellis]

I am away from a Windows 10 computer for the rest of the summer, and then I have to get back to teaching High School so I may not be doing much on this thread. If anyone (Steven?) wants to test out the github site at 

https://github.com/hpssjellis/my-installs-for-bash-on-windows

and install it using the provided instructions, I will still keep updating the site if people want to make Pull Requests (PR), I will just will not be able to test it myself.

Presently I have everything sort of working except updating the Android SDK (Not needed for Magenta but really needed for lots of other developer stuff) and installing Oracle java 8. I have managed to install java 8 just not the oracle version that Magenta wants.

On my computer the install takes about 4 hours with lots of entering your sudo password. I am trying to make that easier.

[Peter Newell]

Thanks for taking a stab at this!

I started the install around 9:20 on my local machine, just happened to see a message about cloud 9 not giving access to the tensorflow directory, so making a copy but it doesn't matter since it's not our machine anyway.

I had previously installed tensorflow already on my bash on windows, so I'm guessing there's an assumed directory structure (since it was already installed and we're sudoing so there shouldn't be permissions issues). 

This is also great for helping me remember what I set my password to on bash.

I'll update again in a bit, it's still rolling along, now refreshing sources on the android repository.

With Gratitude,

Peter

--
Magenta project: magenta.tensorflow.org
To post to this group, send email to magenta...@tensorflow.org
To unsubscribe from this group, send email to magenta-discuss+unsubscribe@tensorflow.org
---
You received this message because you are subscribed to the Google Groups "Magenta Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to magenta-discuss+unsubscribe@tensorflow.org.

[Peter Newell]

It got hung up on Validating the XML when refreshing sources on android, eventually crashing java, but then the process picked up and finished.

In case it wasn't clear, I started from master, not the branch you have been working on.

I'm running the test-setup.sh from master now, it seems to be redoing a bit of what was already done, but maybe it'll be a little more successful this time (back to Validate XML step that failed previously).

export paths to the .profile file so other terminals can use android sdk

export paths here so this bash file can use android sdk

Android sdk installed now you have more to do.

This works best using bash from the power shell

Each line needs a fresh power shell

probably best not to do the echo y |

installing android sdk update

Running as sudo su but in the home folder of pnewell

bash: line 3: android: command not found

bash: line 4: android: command not found

bash: line 5: android: command not found

bash: line 7: android: command not found

bash: line 8: android: command not found

bash: line 9: android: command not found

Refresh Sources:

  Fetching https://dl.google.com/android/repository/addons_list-2.xml

  Validate XML

I don't need android in the immediate term, so if this crashes java and continues the same path again, I'll just continue on testing magenta.

Again, thanks for this, I see the scripts you have setup in the mymagenta directory, it's pretty much what I hoped to have time to do at some point in my hacking, but hadn't made enough time for nice things like structure and repeatability.

With Gratitude,

Peter

[Peter Newell]

Alas bash a01-rnn_basic.sh has a whole lot of not finding bazel, I'll share my process and outcomes in case someone finds it useful.

I look at the log from the finish of the test_setup.sh and see

Now checking if Bazel is installed
--------
------------------------------------------------------
.
echo Now checking where bazel is installed
readlink -f
echo
echo --------------------------------------------------------------
echo .

I went up a couple directories, saw bazel directory and ran the bazel installer there and ran it, so it now sees bazel.

~/mymagenta/magenta$ sudo bash a01-rnn_basic.sh
Extracting Bazel installation...
.
WARNING: Sandboxed execution is not supported on your system and thus hermeticity of actions cannot be guaranteed. See http://bazel.io/docs/bazel-user-manual.html#sandboxing for more information. You can turn off this warning via --ignore_unsupported_sandboxing.
INFO: Loading package: @local_jdk//

stuck here for the moment, but feeling good about the progress, feel like I'm already pretty close to fully functional in less time than expected! Thank you.

Still stuck a good bit later, going to watch the Perseids.

[Peter Newell]

Came back to a BSOD that seems to be unrelated, was ultimately resolved by unplugging some USB devices.

Now that I've had a chance to get back to testing the magenta install, bazel said it had a corrupt installation, which had me remove a folder. Subsequent runs then aren't detecting Java. Switching to java 8 fails java -version, Java 7 seems to work okay, but we need 8.

That said, it looks like there's a fix in the insider build (Fast ring 14901) and Ben Hillis is working on getting it promoted (https://github.com/Microsoft/BashOnWindows/issues/746).

Will follow up again once java is working.

With Gratitude,

Peter

[Jeremy Ellis]

Steven or Peter when I get back to my machine I will try upgrading to Ubuntu 16.04 and then try installing java8 and Bazel then Magenta

If you want to give it a go I have a bash file setup at

https://github.com/hpssjellis/my-installs-for-bash-on-windows/blob/master/a30-upgrade-to-ubuntu-16.04.sh

[Steven Lamphear]

Thanks for all of your work here, Jeremy! Your scripts worked to get Java and Bazel installed, though for some reason I had to update the path again for bazel commands to be recognized. After doing that, I was finally able to successfully execute "java -version" and "bazel -h" commands, which is more than I can say for any of my previous attempts (all right!); however, trying to run the Magenta tests still result in a hanging system. I think my best plan of action at this point is to just keep using my trusty Ubuntu VM and try this all again in a few months. Hopefully that will give Microsoft enough time to make some updates to Bash for Windows!

Thanks again!

-Steven

[Jeremy Ellis]

I agree Steven. I think waiting for another version of bashOnWindows makes sense. I can install most things but the important ones Java8, Android and Bazel have issues. I did get Ubuntu 16.04 working but it does not seem to help with the above issues, so I think I will wait.

Working with bashOnWindows has reminded me how much I enjoy using a cloud service like Cloud9 http://c9.io since I do not have to be stuck at one computer. Unfortunately Cloud9 does not assign enough ram (free site 1Gb, paid site 1.25 Gb) to make Magenta much fun. Tensorflow works fine there. My repo takes a while to install but worked last week :> https://github.com/hpssjellis/forth-tensorflow  Still hoping I can find a cloud service that allows free public sites, that runs Magenta either from a repository or docker. 

Just saw that docker has a cloud site, but I seem a bit lost there. Not sure about Google Cloud, wonder if some of the Google employees can comment on that.