How to connect devices on another host to the one running stf?? Please help...

[Alfred Xia]

Hello everyone,

I am currently trying out STF, and I'm having issues which I'd like to hearing from you experts.  Any input would be appreciated!

Here's my situation:

* 2 Ubuntu Server 14.04 (let's say their ip address is 10.241.54.10 and 10.241.54.16 respectively)

* sudo ufw status shows "Status: inactive" on both 2 Ubuntu servers

* I did not use the docker solution, instead I installed stf and all its dependencies on both 2 Ubuntu servers

My steps/cmds to run stf:

on box1:

* start rethinkdb

* stf local --allow-remote --public-ip 10.241.54.10 --provider 10.241.54.10 --provider-min-port 15000 --provider-max-port 20000

on box2:

* start rethinkdb

* stf local --allow-remote --public-ip 10.241.54.16 --provider 10.241.54.16 --provider-min-port 20000 --provider-max-port 25000

Then I can successfully see devices showing up from http://10.241.54.10:7100/#!/devices and http://10.241.54.16:7100/#!/devices

My question:

I want to use box1 (54.10) as the master box, and connect box2(54.16) to box1, so from http://10.241.54.10:7100/#!/devices, I can see all the devices connected to both box1 and box2.

I guess I should start provider on box2, but what exactly the parameters I should use to connect to box1?  Please help!

I tried as below:

stf provider --allow-remote --name 10.241.54.16 --min-port 20000 --max-port 25000 --connect-sub tcp://10.241.54.10:7114 --connect-push tcp://10.241.54.10:7116 --storage-url http://10.241.54.10:7100/

but it shows:

INF/provider 11938 [*] Providing all 0 of 5 device(s); ignoring "0123456789ABCDEF", "284cd8fe", "639aa439", "YLHU657DMRG6NNN7", "7TTKM78PNFS8SW7D"

Thanks,

Alfred

[fba]

Hi Alfred,

Generally, it is best to use the dockers.   Docker has a bit of a learning curve, but isn't too bad once you have spent a few (long) hours with it.   You also get the added bonus with the dockers that when you restart the services, you will get the latest release version.  (Assuming you are configured that way.)   But, perhaps the best reason to use the dockers is you are more likely to get useful help from other people here on the lists.   There aren't many people that are really qualified to help with mixed "local" and remote installs.  (That includes myself, but I hope to be able to help a little anyway.)   If you go to the deployment doc (link below) and just copy/paste the systemd scripts, change the IP addresses to what you use, and start them up.   You'll have a fully functional OpenSTF server pretty easily and quickly.   (That said, I just re-read your post and saw you are on Ubuntu 14.04, so systemd may not be an option.   I have OpenSTF working really well using Docker and systemd with Ubuntu 16.04, so if upgrading is an option, it is probably worth it.)

It looks like you are part way to solving your own problem.   However, I question the ports that you are using.   If you use the stf-provider systemd script at :

https://github.com/openstf/stf/blob/master/doc/DEPLOYMENT.md#stf-providerservice

It would indicate that the port for --connect-sub should probably be 7250, and the port for --connect-push should probably be 7270.   That script also doesn't have a port defined for the --storage-url, so you may want to strip that off.   You might also run "netstat -l" on 10.241.54.10 to be sure that the ports that need to be listened on, actually are being listened on, and to verify that you have the correct port numbers.   Also note, that from what I can tell, there isn't really any kind of handshaking that takes place to let you know if you are connecting to the wrong ports.   There also doesn't seem to be any kind of error messages logged when you have the wrong IP address or port number.   You basically have to try to figure it out on your own.   I found that in some cases you may need to bust out a sniffer in order to figure out what is really going on.

For the provider showing that it is providing 0 of 5 devices, you may have to touch each device an allow the developer key from the provider on each one.   If you have already done that, then you may need to either dig in to the docker that provides adbd, to see how it is set up, or the source to see how it is used.  (I have not dug in to that part at all, but probably will soon!)

Hope this helps.. At least a little. ;)

[Alfred Xia]

Hi fba,

Thank you so much for all the explanation and recommendation.

I am not quite sure if I am able to set up docker on our Ubuntu servers, but I may give it a try.

Do you have any documented steps/instruction to refer for a newbie?  That would be definitely a great help.

Anyway, thanks again.

Alfred

[Alfred Xia]

Hi fba,

I am not sure what topology you are using with docker solution for openSTF, but can you please shed some light on following:

Assuming you have 2 hosts, A and B, and you have different set of devices connected to each of the host.  Then how to set it up with docker?

From my understanding, one host should be app side, let's say A; and B will be the dev side.  

According to deployment doc on github, I guess app side is served ONLY as app, no provider running on it, right? ONLY running provider on dev side, right?

So, if you want to run app side on A, dev1 on A, dev2 on B, what additional units you would start?

Sorry if it's kind of confused, hope you can understand what I am saying :)

-Alfred

On Monday, August 29, 2016 at 11:43:12 PM UTC+8, fba wrote:

[fba]

Hi Alfred,

Getting your head around everything with OpenSTF will take some time and work.   And, the available documentation, while good, doesn't seem to be targeted toward people without a lot of experience with networking, docker, Linux, nginx, and Android development.  I had no docker experience when I started to try to put together my OpenSTF system, and originally tried to set it up using the "local" command.   I quickly ran in to all kinds of weird problems, which ultimately led me to bite the bullet and just set it up using the documentation on the web site.   I've been meaning to write a blog post about how I set things up, but I have been entirely too busy with a break-neck development schedule here at work.

So, let me see if I can share a little bit of hard-earned wisdom about OpenSTF, with the hope that it helps you out.

First, the documentation is really geared toward setting up OpenSTF as a large cluster of machines.   I suspect that if you have a lot of devices, and lots of people trying to use them at the same time, that using lots of machines would be a good idea.   However, my initial setup for testing involved about 3-4 dozen devices, and only myself using them all.   I was able to run a decent test setup like that on a small low-profile Dell machine with a Core i3 @ 3.3Ghz and 8 GB of RAM.   It wasn't blazingly fast, but it work well enough for my testing needs.

Next, it is important to understand that *MOST* of the "pieces" of OpenSTF can run wherever you want them to run.   They can all run on the same box, or they can (mostly) each run on their own box.   There are a few exceptions, but those our outlined in the documentation.  However, the documentation seems a little scary at the beginning.   But, once you start to get the feel of how everything works, it starts to become more approachable.

Next, it is worth pointing out that there are a LOT of dependencies to get OpenSTF up and running.   I've tried to get them one-at-a-time and get it all going, and it was horribly painful.   Along the way I ran in to issues with incorrect versions of various things when the Linux version I was using didn't have the exactly correct version of a library or tool.   The dockers, while a little scary at first, completely remove the need to figure all of those dependencies out.  When the docker container is downloaded to your machine, everything that is needed to run the system is already there!   Even better, if you build your systemd startup scripts properly (which is the default from the documentation), you will get the latest release builds of OpenSTF any time you reboot the machine, or restart the systemd services!

Finally, it really isn't that hard to get things up and running if you use the available documentation, and a Linux system that can run docker and uses systemd.   I'm running it on Ubuntu 16.04 server.   I do *STRONGLY* recommend that you use a version of your favorite Linux distro that can easily install things like docker, and that already has systemd integrated and running.   It will save you a *LOT* of time and pain.

But, on to my configuration.  I currently have two OpenSTF instances running.   The one in my home office is running everything *BUT* the provider service in a VM on a rather beefy Dell server.   The provider is running on the previously mentioned low profile Dell desktop machine.   Both machines are in the same layer 2 domain, and on the same IP subnet.  (One at 192.168.64.55, the other at 192.168.64.56.)

However, as I said, I originally ran everything on a single box.   If the documentation at https://github.com/openstf/stf/blob/master/doc/DEPLOYMENT.md makes you feel like an anxiety sufferer on a sugar bender, this is probably a good way to start out.  It'll let you get everything working, and then allow you to branch out slowly from there.   If, as I suggested, you are running a Linux distro that is current enough that you can "apt-get" (or yum, or whatever) docker, and that is running systemd as the default mechanism, then pretty much all you need to do is install docker ("apt-get install docker.io", FWIW), then copy/paste all of the .service files (that *AREN'T* listed as optional) in that document, put them in /etc/systemd/system (or whever your OS of choice puts them), change the IP addresses used (and *ONLY* the IP addresses!  If you mess with the port numbers you will create a LOT of pain for yourself!) and then start them up with something like "systemctl start <service name>".

With a few minor exceptions.   The systemd scripts that end with @.service need you to start them up with a parameter.   So, you would use "systemctl start <service name>@<parameter>".  (You can generally leave off the .service portion of the file name.  Systemd is smart enough to figure that out.)  The parameters to use appear to be somewhat arbitrary, but to keep myself sane, I used the values that were in the documentation for each of the .services files.  (i.e.  stf-app@3100, etc.)

Once you have all of those systemd service files on your system, you may want to have them run at startup.  To do that, I added the following lines to the bottom of each service file :

[Install]

WantedBy=multi-user.target

And then used "systemctl enable <service-file>@<param>" for each one.   From then on, when the machine boots, it should start those services.

Do note, that the first time you start the services, it might take a while to start.  It will be going out and downloading the docker containers for each of the pieces.  If you network connection is slow, you might have to wait a bit.  But, after that first download, it starts up pretty fast.

Once you have everything running on a single machine with those service files, you can start to branch out and move services files to other machines.  (Or, add other machines to the cluster.)    For my current two machine setup, I put the adbd service, and the stf-provider@ service on the second machine, and edited the service file to use the VM instance's IP address to connect to the cluster.

I think the only other thing that messed me up was the nginx configuration.   When you bring up the stf-provider container, you will end up using a parameter like "01".  In the nginx configuration file, you will see a configuration block like this :

    # Handle stf-pr...@floor4.service
    location ~ "^/d/floor4/([^/]+)/(?<port>[0-9]{5})/$" {
      proxy_pass http://192.168.255.200:$port/;
      proxy_http_version 1.1;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection $connection_upgrade;
      proxy_set_header X-Forwarded-For $remote_addr;
      proxy_set_header X-Real-IP $remote_addr;
    }

On the "location" line, you will see the text "floor4".  Whatever parameter you use when you start stf-provider@ needs to go where the "floor4" text is.  And, you need a block like the one above for each provider that you have running.

So, in the end, probably the best way to get going is just to start with that deployment file and jump in.   Then, when you hit snags you can ask smaller, more simplistic questions along the way and it will probably be easier to help.

Good luck!

[Alfred Xia]

wow, that's really a long and detailed kind of experience sharing and beginner's guide to OpenSTF :) !  Thank you for your help REALLY!!

While I was struggling really hard to try to get it work with only stf local solution, I started to realise that I was walking in a tough road which anyone else will not do the same.  So I've decided to go with docker way.  First I need to talk to our ops guys if it's OK to upgrade Ubuntu server from 14.04 to 16.04. If it's OK, I will try to start it all over again according to the deployment guide, and of course your GUIDE in case of any confusions.

Thank you very much again!

BTW, is there any other way to communicate with you, or you prefer reply right here?  I just don't want to lose connection if you stop following this google group or too busy to check in :)

-Alfred

[Alfred Xia]

Hi fba,

I switched to docker solution, but still a lot of issues, really need your help here.

So far, I can start nginx stf-auth rethinkdb-proxy-28015 successfully, but cannot start rethinkdb stf-app successfully. Not tried for others yet.

for rethinkdb, I cannot start it with "systemctl start rethinkdb.service" (you can see more details @ https://github.com/openstf/stf/issues/415), but I can start it by running cmd:

/usr/bin/docker run --rm --name rethinkdb -v /srv/rethinkdb:/data -e "AUTHKEY=YOUR_RETHINKDB_AUTH_KEY_HERE_IF_ANY" --net host rethinkdb:2.3.5 rethinkdb --bind all --cache-size 8192

[Alfred Xia]

Hi fba,

I finally managed to start STF successfully with docker solution, really thank you for all your help!!

Please consider this thread as completed, I will ask questions in separated threads.

Thanks,

Alfred

[habeeb shk]

Hi Alfred, 

 i am working with openstf using docker as well and ran into some trouble installing it. I could not connect remotely using the stf provider, so i was thinking if you can help me through it or if there is any kind of documentation for it. thanks