Developing managed Experiments

[Johanna Roussel]

Hey,

after some time has passed I'm now focusing again on the remote lab development and I want to make sure if I understand some parts correctly:

Is it correct, that you are using the unmanaged approach for the aquarium lab and the managed approach for the archimedes lab?
Are you using the non-python managed servers for the archimedes lab? I looked into your archimedes.py but I did not understand how it communicates with the "boards". What pieces of hardware are you using? And are there further software modules in the backend?

It's not yet clear how the remote labs at our university will look like - but surely we will measure sensors and control devices. For that we will use a raspberry PI or a Microcontroller for each experiment.
Under the assumption I will develop the experiment servers in python using the managed approach:

If using the Raspberry Pi.. is the preferred approach to develop a non-python managed server (XML-RPC approach)? Meaning that each Experiment Server is running on a Raspberry Pi? For that I have to use: http://weblabdeusto.readthedocs.io/en/latest/remote_lab_development.html#remote-labs-development-managed-server-python ? What is the reason that I have to use this approach and not http://weblabdeusto.readthedocs.io/en/latest/remote_lab_development.html#remote-lab-development-weblab-python ?

Using a Microcontroller for each experiment and under the assumption having a few labs, also the XML-RPC approach makes sense, right? Because Core and Lab process always have to run on the same host?
The infrastructure could look like this:
-------------------------------------------------------------------------------------------------------------------------------------------------------------
Host1:
core_host    -->   lab1_host
                   -->   lab2_host ...

Host2 of Lab1:
multiple experiment servers running on host2 (different microcontrollers connected to host2 for the different experiments)
-------------------------------------------------------------------------------------------------------------------------------------------------------------
In that case I could use http://weblabdeusto.readthedocs.io/en/latest/remote_lab_development.html#remote-lab-development-weblab-python ? "while the first one could be more convenient for regular deployments" is written in the documentation. But I would need launchers as well, right?


My thought was, that it doesn't make sense to manage all physical labs with only one host.
I hope it's not to confusing.

Best,
Hanna

[Pablo Orduña]

Hi,

On Fri, Jul 29, 2016 at 8:26 AM, Johanna Roussel <hann...@gmail.com> wrote:

Is it correct, that you are using the unmanaged approach for the aquarium lab and the managed approach for the archimedes lab?

It is correct.

Are you using the non-python managed servers for the archimedes lab? I looked into your archimedes.py but I did not understand how it communicates with the "boards". What pieces of hardware are you using? And are there further software modules in the backend?

The archimedes.py code is in a regular server and it communicates with several Raspbery Pis (one per ball). Each ball is contained in a box. Here you can see a picture of how it looks: http://weblab.deusto.es/pub/archimedes_picture.jpg

In the code of archimedes.py in _send you can see how it calls the board location (e.g., different IPs):

     https://github.com/weblabdeusto/weblabdeusto/blob/master/server/src/experiments/archimedes.py#L238

In the raspberry pi some code that uses GPIO is run:

     https://github.com/weblabdeusto/experiments/tree/master/firmwares/archimedes

I'm not sure whether is one of these two files or if they're updated, but it may help as for having a general idea of how it works:

     https://github.com/weblabdeusto/experiments/blob/master/firmwares/archimedes/levelSplash.py

     https://github.com/weblabdeusto/experiments/blob/master/firmwares/archimedes/ServerSplash.py 

As you can see, these are just an auxiliar weblab-independent code and it's the weblab-dependent code (i.e., archimedes.py) the one indirectly using it. So in this case we have 1 regular server running the archimedes.py and it connects to 6 raspberry pis.

It's not yet clear how the remote labs at our university will look like - but surely we will measure sensors and control devices. For that we will use a raspberry PI or a Microcontroller for each experiment.
Under the assumption I will develop the experiment servers in python using the managed approach:

If using the Raspberry Pi.. is the preferred approach to develop a non-python managed server (XML-RPC approach)? Meaning that each Experiment Server is running on a Raspberry Pi? For that I have to use: http://weblabdeusto.readthedocs.io/en/latest/remote_lab_development.html#remote-labs-development-managed-server-python ? What is the reason that I have to use this approach and not http://weblabdeusto.readthedocs.io/en/latest/remote_lab_development.html#remote-lab-development-weblab-python ?

No, it's not a big deal. The main thing is that WebLab-Deusto is not particularly small in size (now it is much more than it used to be with the old client), so using it would consume quite a lot of space in disk. But both approaches should be fine, so pick whatever is easier for you.

 

Using a Microcontroller for each experiment and under the assumption having a few labs, also the XML-RPC approach makes sense, right? Because Core and Lab process always have to run on the same host?

Yes, even the Laboratory Server and the Core server are often in the same host (and then the experiment servers in constrained devices). You may have for example a core server and lab server in a machine, and then one raspberry pi or microcontroller per experiment. In the case of archimedes, for example, you can even have all the code (core, lab, experiment) in the same host, and the experiment is the one that connects to the multiple raspberries/microcontrollers. But the most typical one is that in one proper host (a computer or server) you have the Core and Laboratory and then each experiment hosts an experiment server in whatever device.

 

The infrastructure could look like this:
-------------------------------------------------------------------------------------------------------------------------------------------------------------
Host1:
core_host    -->   lab1_host
                   -->   lab2_host ...

Host2 of Lab1:
multiple experiment servers running on host2 (different microcontrollers connected to host2 for the different experiments)
-------------------------------------------------------------------------------------------------------------------------------------------------------------
In that case I could use http://weblabdeusto.readthedocs.io/en/latest/remote_lab_development.html#remote-lab-development-weblab-python ? "while the first one could be more convenient for regular deployments" is written in the documentation. But I would need launchers as well, right?

I'm not following this point. That comment referred only to the disk space consumption (and, in the past, it was even worse with the old client: when you run python setup.py install, it would try to compile in Java so many things that a Raspberry Pi was not enough). I should update the docs to reflect this. 

My thought was, that it doesn't make sense to manage all physical labs with only one host. 

I hope it's not to confusing.

 

I'm afraid I'm not following this part either. The load of users is managed by the Core server. If there are 200 students accessing weblab deusto, only the core server will be stressed. But the laboratory server or the experiment server will not receive anything other than the current active students. It is very typical to have few laboratory servers (few load on them).

Best,
Hanna

Best,

--

Pablo Orduña ( http://morelab.deusto.es/p/pablo-orduna/ )

MORElab - Envisioning Future Internet (http://morelab.deusto.es)
WebLab-Deusto - (http://weblab.deusto.es)

[Johanna Roussel]

Thank you - ServerSplash.py was useful to understand how it works!

In your case archimedes.py (the experiment server) is running on the same host as the whole weblab framework. The configs concerning the board locations are stored in the configuration.yml file?

If I want archimedes.py to run on another host, I would use the xml-rpc approach and the simple script approach (http://weblabdeusto.readthedocs.io/en/latest/remote_lab_development.html#remote-labs-development-managed-server-python). I was confused because I misunderstood this sentence: "The second one would be recommended for constrained devices (e.g., Raspberry Pi)".
I didn't associate the "Raspberry Pi" in the sentence with the location of the weblab framework - meaning I assumed that the weblab framework is not hosted on a raspberry pi - but maybe raspberry pis are used for the experiments. But the experiment server could also be located on a standard PC (not the one where the weblab framework is located). But in that case the xml-rpc + simple script approach is needed, as well. So - raspberry Pi (relating to the experiments) is not the reason that the simple script approached should be used.

I was confused, because I thought the experiment server is doing a lot more than it actually does. But the work is handled by further server running e.g. on raspberry PIs belonging to the experiments. From that point of view it's okay that the experiment server is in the same host as the weblab framework.

Best,
Hanna

[Pablo Orduña]

Hi,

On Mon, Aug 1, 2016 at 2:12 AM, Johanna Roussel <hann...@gmail.com> wrote:

Thank you - ServerSplash.py was useful to understand how it works!

In your case archimedes.py (the experiment server) is running on the same host as the whole weblab framework. The configs concerning the board locations are stored in the configuration.yml file?

I was checking how it is deployed now, and yes, it's in the same host, and the board locations are actually specified in another file pointed in the configuration.yml file:

          archimedes:

            config_file: plunder/general_laboratory/archimedes/server_config.py

            type: experiment

            class: experiments.archimedes.Archimedes

and then in that file:

#!/usr/bin/env python

#-*-*- encoding: utf-8 -*-*-

archimedes_instances = {

    "archimedes1": "http://192.168.0.161:80",

    "archimedes2": "http://192.168.0.162:80",

    "archimedes3": "http://192.168.0.163:80",

    "archimedes4": "http://192.168.0.164:80",

    "archimedes5": "http://192.168.0.165:80",

    "archimedes6": "http://192.168.0.166:80",

#    "archimedes7": "http://192.168.0.167:80"

}

archimedes_real_device = True

 

If I want archimedes.py to run on another host, I would use the xml-rpc approach and the simple script approach (http://weblabdeusto.readthedocs.io/en/latest/remote_lab_development.html#remote-labs-development-managed-server-python).

If you're using the version in that doc section, yes, it must be XML-RPC. If you're using this other one: http://weblabdeusto.readthedocs.io/en/latest/remote_lab_development.html#remote-lab-development-weblab-python

then you can leave without specification the protocol and the system will use whatever suits best (which will be HTTP+JSON+pickle object right now).

For example, in the past, we had archimedes.py running in yet another server (one we use for developments still under test and with more people having access to it), not using XML-RPC, but then we moved it to the main server. The point is that it doesn't really matter where it is, and by changing the configuration.yml to move it and the assigned address in the laboratory server, it works. 

 

I was confused because I misunderstood this sentence: "The second one would be recommended for constrained devices (e.g., Raspberry Pi)".
I didn't associate the "Raspberry Pi" in the sentence with the location of the weblab framework - meaning I assumed that the weblab framework is not hosted on a raspberry pi - but maybe raspberry pis are used for the experiments. But the experiment server could also be located on a standard PC (not the one where the weblab framework is located). But in that case the xml-rpc + simple script approach is needed, as well. So - raspberry Pi (relating to the experiments) is not the reason that the simple script approached should be used.

Yes, but they're two different but too related things: weblab-deusto "deployed" and weblab-deusto "installed". I mean, at this moment, for the sake of simplicity, we don't have a "weblab-deusto library": it's just the same project as the rest of the software (e.g., the core server). So when you are going to use in a Raspberry Pi the experiment server using all the "from weblab.experiment.experiment import Experiment" code, you still have to download the git repository, run the "python setup.py install", etc., which will install the code of everything (e.g., the core server, the lab server, the experiment servers etc.), even if you never use it because you're using a regular PC for the Core Server + Lab Server. I mean, the software is still installed (and requires space). Dependencies are not required (e.g., you don't need MySQL, Redis, Apache, nginx or whatever). Right now it is not a big problem (WebLab-Deusto without the old client is far smaller; it use to require like 4 GB RAM for compiling it + long time + hundreds of MB of space), but still, sometimes it's like having a still big system for something so small that it doesn't make that much sense. But it's still perfectly fine to use it.

 

I was confused, because I thought the experiment server is doing a lot more than it actually does. But the work is handled by further server running e.g. on raspberry PIs belonging to the experiments. From that point of view it's okay that the experiment server is in the same host as the weblab framework.

That depends on the experiment. In the case of the archimedes, the problem is that we have several raspberry pies, so we need another software server connecting to them all. This software server does not need any hardware, so it's fine to put it with the rest of weblab (or somewhere else as it used to be). But if the experiment had a single raspberry, it would actually be better to have the code running there.

Hope this helps,

--

You received this message because you are subscribed to the Google Groups "weblabdeusto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weblabdeusto...@googlegroups.com.
To post to this group, send email to weblab...@googlegroups.com.
Visit this group at https://groups.google.com/group/weblabdeusto.
For more options, visit https://groups.google.com/d/optout.