Install Spark and Zeppelin Image though Docker on Windows 10 with Virtual Box: Errors / TroubleShoot

[r poon]

Background:

I have to deploy the Docker software to my existing Windows 10 PC (with Virtual Box 5.2.x installed) so that I can download a Spark 2.x docker image (from IBM BigDataUniversity)

I am new to Docker  and I though my six hours experience will be useful for those facing similar situations. 

I have browsed through the Docker web which indicated that Docker Toolbox is for legacy systems (supposed including Windows 10 without Pro or Enterprise)

I downloaded Docker Toolbox for Windows (patched v1.8.1d) as suggested 

Trouble Shoot 1 (Existing Virtual Box Corrupted after Installing Docker):

During the Docker installation process,  I tried to include (as an option from the Docker ToolBox installation menu) the auto-install of another Virtual Box (of version 5.0) come with Docker ToolBox 1.8 to my PC: Since I read from web sites there were many complaints about existing VB not being recognized, I decided to try to include the Virtual Box.

Problem - Even though I specified to another E:Programs location and thought there will be side by side installation of new and old version of Virtual Boxes, I was wrong, and the installation eventually CORRUPTED the Virtual Box .

Solutions - I have to remove the i) Virtual Box 5.0 and ii) reinstall Virtual Box 5.2, iii) Re-install Docker

TroubleShoot 2  (Short cut to Quick Start has no response)

After I corrected a Git Bash path default (in the properties field of the Quick Start Icon) which caused the initial problem, there was another symtom that the screen simply flashed for a split second and did nothing. I found out the docker-machine.exe actually get stucked. This points to a deeper access problem below

Trouble Shoot 3 (Docker ToolBox Version 1.8 TCP Port Problem - cannot access default VB properly):

Since my notebook has no Windows 10 Pro installed, so I am stucked to Docker ToolBox (instead of the full blow Docker for Windows). However the v.1.8.1d come with a notorious problem of TCP port problem, 127.0.0.1: causing errors in the access paths to the VMs inside Virtual Box

Without resorting to the more tedious firewall pots opening, I tried to use V1.9 of tool box which "solved" the TCP port problem

Challenge 4: http://xxxxxxxxxxxxxx: 8080 (Zeppelin Notebook) cannot be accessed (New and Old Version of VM/

Container): 

Docker - Error response from daemon: client is newer than server

However another error message (that client is newer than server )- I am not exactly sure about the meaning on day 1 of my Docker installation.

After search around, I tried to "update" the default VM (get installed by Docker ToolBox into Virtual Box as part of the Docker ToolBox installation.

This one could be due to the v1.8 docker installation but I am not 100% sure whether in each Docker ToolBox installation it install a new default.

(Again this default also causes various problems, many has to remove the existing default VM and re-install another default VM )

 - this would probably due to solve the port pass-through problem). 

Solution: Update the docker VM from Gitbash  - $ docker-machine upgrade default

I also noted this is a boot2docker version of mini-linux that I have to further studied to see what it is like