Updated - Wifi ClusterHat setup tutorial (DHCP, DNS, Ansible, Docker, and hostapd)

[Robert Metcalf]

Hi,

A number of people have given me feedback on the tutorial I posted regarding setting up a portable demo cluster for demoing docker, ansible etc.

(https://code2.metcarob.com/node/300)

Since there were reports of problems due to new versions of software I bought myself a new set of SD cards and reworked through the tutorial.

I found many bugs which I have corrected, I then went through a third time.

Thank you to everyone for the feedback.

Robert

[Alexis Iglauer]

Thanks, looks great, and it feels super pedantic to make this comment but it's guineapig (not gunieapig).

Thank you and regards

Alexis

[Scott Beeker]

Robert, Thanks !  I did get the old tutorial parts 1-4 to work...  Will do it again with a new master rpi3 image.

When I click on your link it takes me to the old tutorial with old dates on it.  Is there some change I can look for to be sure I am getting the latest. The link below is taking me to what looks identical to previous so wanted to be sure I got the latest.  The dates did not change apparently or I am on the wrong link.

If there is just one last change I can identify to be sure it is latest.

[Scott Beeker]

Robert, FYI, When you go to your sharepoint site many warnings from web sight protection like Norton etc for whatever "driveby downloads" are.  Two machines I tried one with Norton and one with Kapernisky web protection complain about "driveby downloads" saying your site is malicous now. It was not doing that a month ago or so.

On Sunday, May 14, 2017 at 9:54:02 AM UTC-4, Robert Metcalf wrote:

[Robert Metcalf]

Scott,

Thanks for the report.

I went through the Norton site owners process last night and they have re-run their analysis my site and are now reporting it as ok

https://safeweb.norton.com/report/show?url=metcarob.com

It's now showing as safe. (I haven't changed anything at all.)

Robert

[Bill Corrigan]

Hi Robert, 

Great tutorial, I have really enjoyed and learned a lot going through it. 

I've reached a point where I'm stuck and can't find the right information anywhere to get me back on track.

In:

https://code2.metcarob.com/node/298

Step 21 - Deploy docker containers to nodes in the cluster,

Install apachepi.service file does not execute. 

I believe it is because it can't find a file. The output is:

TASK [Install apachepi.service file] **********************************************************************************************************************

fatal: [p1.bcorrigan-local.com]: FAILED! => {"changed": false, "failed": true, "msg": "Unable to find 'docker_any.service' in expected paths."}

the code in the .yml file is:

- name: Install apachepi.service file
    template:
      src: docker_any.service
      dest: /etc/systemd/system/docker_{{ service.shortname }}.service
      owner: root
      group: root
      mode: "u=rwx,g=rwx,o=rwx"

Where would the absolute path to that file be? 

Thanks for your help,

Bill

[Robert Metcalf]

Hi,

Glad you enjoyed the tutorial.

You have discovered a big hole in the Ansible part. I failed to describe the creation of the required files.

I have updated the page with instructions of creating install_docker.sh and docker_any.service

https://code2.metcarob.com/node/298

These go in the ~/ansible directory which I have put into git for reference: https://github.com/rmetcalf9/metcarob-local_cluster

Thanks for the feedback, let me know how you get on with the rest of the tutorial

Robert

[Bill Corrigan]

Thanks that was the needed pice of information. I am now through pt. 4.

Thanks again, great work, good style, easy to follow. 

I think for pt 5, I will digress a bit. I have RPi Zero W, and so I think I will make that the Access Point. I will attempt to set up the WAN on the USB port (which I can hook to my laptop, and the WiFi as the WAP, for the cluster to get networked. I'll post here how it goes. 

Thanks,

Bill

[Bill Corrigan]

Hi Robert, 

I went ahead with pt 5 successfully and want to thank you for all of this. I gained a lot going through it. 

One thing that isn't working quite right is name resolving from my Mac client to the p1 and p2 web servers. I can get out to the internet (I'm doing that now via the Pi_Cluster_Admin WAP as I type here) and I can do the wget from the controller to p1 and get the index.html file returned, but on my Mac, http://p1 and http://p1.bcorrigan-local.com don't resolve. Using the IP address of p1 and p2 does work. 

I tried adding the controller Ip address, 192.168.2.1 and the private one that showed up running ifconfig on the controller for wlan1 and wlan0 respectively, but no luck. 

On Sunday, May 14, 2017 at 8:54:02 AM UTC-5, Robert Metcalf wrote:

[Robert Metcalf]

Hi,

Glad it's almost working for you.

I assume the situation is that looking up http://p1 and http://p1.bcorrigan-local.com work from the cluster controller, but not from your mac connected to the cluster controller.

Since you can get to the internet I assume that looking up www.google.com is working from the Mac.

This suggests to me that your mac is not using the DNS server that it should be.

1. Can you verify the DNS server the mac is using. (I am not a mac user so I don't know the command for this)

If the mac has the wrong DNS server configured then the DHCP server on the cluster controller must not be giving out the correct DNS server address.

2. Can you look up  http://p1 and http://p1.bcorrigan-local.com from p2, p3 or p4?

If you can it means that p2, p3 and p4 are getting the correct DNS settings.

I had a few problems with this. Some problems were caused by my getting erroneous results. My ISP had set up their DNS server to give me a valid response when I looked up invalid addresses. This is so that if I am using a browser they can give me adverts but it's annoying from the command line because I spent ages wondering why I can ping p1, p2, etc. but not SSH. (I was pinging the ISP's server without knowing it)

Let me know if what you are seeing corresponds to any of the above and maybe post your dhcp server config file. 

Robert

[Bill Corrigan]

Hi, Robert,

Yes it is the Mac where the issue is. Things work fine on the cluster, I can get results running wget http://p1 from p3 and p4. 

On the Mac, I run: scutil --dns | grep 'nameserver\[[0-9]*\]'

Which returns:

  nameserver[0] : 8.8.8.8

  nameserver[1] : 8.8.4.4

  nameserver[0] : 8.8.8.8

  nameserver[1] : 8.8.4.4

  nameserver[0] : 8.8.8.8

  nameserver[1] : 8.8.4.4

  nameserver[2] : 192.168.2.1

So I must be using the wrong name server. When I ping 192.168.2.1 from the Mac, I get a reply so it sees it.

When I do sudo cat /etc/resolv.conf on the controller I get:

nameserver 192.168.2.1 nameserver 8.8.8.8 nameserver 8.8.4.4 nameserver 127.0.0.1 search Home bcorrigan-local.com

So I think I have the correct nameserver in there. This is the IP for br0. Should I be using the IP for WLAN1 or WLAN0?

-Bill

[Robert Metcalf]

Hi,

I am not a dns expert but I think your configuration is wrong.

Your mac seems to have both the controller DNS server and googles default ones.

The mutiple servers are for redundancy not for different servers with different zones.

If DNS server A knows about a domain it will return it. If it doesn't know about the domain it will make a request to it's upstream dns server.

The process continues up the tree until one DNS server is able to provide the example.

When you configured bind on the controller you specified an upstream server. (Either your ISP or 8.8.8.8)

You need your mac to have only the controller DNS server set. If the query is for p1 you will get the local response, otherwise it will go ask it's upstream server.

I think it's working on the controller and not the mac because the controller is trying itself first, but the mac is set to try the controller last. So the mac will only ever ask the controller for DNS lookups if all other dns servers are down.

Robert

[Cameron MacFarland]

Hi Robert.

Thanks for creating this tutorial. It helped me set up things quickly and gave me some ideas on what to install for better networking.

I ended up creating an ansible playbook in order to automate the instructions. It's available at https://github.com/distantcam/ansible-rpi-playbooks

Thanks

Cam

[Robert Metcalf]

Hi Cam,

I just had a look through that playbook. I am very impressed!

There is a lot I need to learn about ansible and that playbook is going to teach me a lot.

(There are a few bits where you configure for Australia which wouldn't work for me in the uk.)

Do you mind if I put a link to it from my tutorial?

Robert

[Robert Metcalf]

One quesiton I have though,

in 

ansible-rpi-playbooks/roles/common/tasks/main.yml

you have a step to enable ssh.

I am surprised to see a step to enable ssh in an ansible playbook since ansible needs ssh to work. So a step to enable ssh seems unnecessary since in order to run ansible at all you must have already had a successful ssh connection.

[Alexis Iglauer]

Hi Cam

That playbook looks great, thanks so much.  On the one hand, the few hours I've put into my own playbook is now for nought, on the other, you've probably saved me (and many others) days worth of effort.

Best regards

Alexis

[Cameron MacFarland]

Yes I need to add some instructions on how to customise it, as currently it has my locale settings and github username.

Feel free to share it around and add a link.

I'm still working out some of the bugs. I just hit an issue with the nodes not networking properly.

[Cameron MacFarland]

Yes ansible needs a ssh connection initially. I'm currently adding a blank ssh file to the boot which enables ssh. I thought that only enables ssh temporarily, which is why I then enable the service fully, but it doesn't seem to be needed. I've left it in for safety as it doesn't hurt to enable an already enabled service.

Also your blog appears to be down, so I can't check to see if I've missed a step.

[Robert Metcalf]

Hi,

A server reboot fixed my blog.

I have added a link to your gitrepo here - https://code2.metcarob.com/node/300

Robert

[Brisbane Stomp]

hello robert, i am about to start your tutorial soon, but me being very new to all this i just want to ask... is this docker setup to parallel process all data via all pi3/zeros in the network? if so can i assume the controller pi3 automaticly uses all zeros processing power at all times? i essentially want to run myrobotlab with machine learning capabilities on my inmoov robot and assume your set up will allow me to parallel process all this via the main controller pi?

[Robert Metcalf]

Hi,

The tutorial just goes through setting up the pi's, installing docker and managing via Ansible.

You could start from this as a base and use it to deploy software onto each of the Pi's which would achieve the automatic processing you are after.
The Pi's and cluster hat are a great platform for learning about this kind of thing but the processing power of the resulting cluster will not be higher than a basic laptop.

Robert

[Brisbane Stomp]

thanks for that. so this setup would allow to run many programs using all cpu's in the cluster. but at a speed no more than 1.2 ghz.

[Robert Metcalf]

This setup will only go so far.

You will have to provide/build Rasp Pi compatible docker containers to run the programs. (You will end up with 4 computers + a controller on which you can run whatever you like.)