Continuous position/stats streamer

[Leonardo Romor]

Hi everyone,

this wants to be a continuation of the discussion started here.

I think that the last Hartley post is a good starting point,

we could use protocolbuffers or grpc to implement the interface.

I will list the stats that in my opinion would be important to show in the form of a proto-like code, just to give an idea

message BgStats { 

string status; // A string or byte that gives the current status of the machine, if it's paused, running, stopped, error(which error?) repeated float pos = 3; // array containing the position in real world coordinates for each axis
float temp; // the temperature thing
float spindle_speed;

repeated float pos; // Real world axes coordinates

repeated float speed; // speed? Or we could derive it outside beagleg

}

and, a command request

message BgCommand { 

string command; // start, stop, pause 

}

-lr

[Leonardo Romor]

Hello, regarding the implementation with grpc, it seems to be a difficult path to merge the async behavior with select() and the async engine of grpc (as pointed out here
https://groups.google.com/forum/#!topic/grpc-io/9LzMrNUTKAM).

So these are the choices imho
a) we go multithread
b) we use select with 0 timeout, (and asyncNext) but in this way we will not exploit anyhow the interrupts
c) we instantiate multiple servers, a udp server for streaming position a tcp one for sending commands and another one to send pure gcode.

do you have other options to add? 
If not what do you suggest? I personally prefer the last one.

-l

[Hartley Sweeten]

On Tuesday, December 20, 2016 4:50 AM, Leonardo Romor wrote:
> Hello, regarding the implementation with grpc, it seems to be a difficult path to
> merge the async behavior with select() and the async engine of grpc (as pointed
> out here
> https://groups.google.com/forum/#!topic/grpc-io/9LzMrNUTKAM).
>
> So these are the choices imho
> a) we go multithread
> b) we use select with 0 timeout, (and asyncNext) but in this way we will not exploit anyhow the interrupts
> c) we instantiate multiple servers, a udp server for streaming position a tcp one for sending commands and another one to send pure gcode.
>
> do you have other options to add? 
> If not what do you suggest? I personally prefer the last one.

Well, option c implies option a. I think the only way to run multiple servers is
to go multithreaded.

Anyway... I like the idea of a udp server for streaming position as well as status
information. Commands, like starting/pausing/stopping/jogging, could also be
received through this server. The current tcp server is used as-is to receive
gcode. I don't think a separate "command" tcp server is needed.

Regards,
Hartley

[Leonardo Romor]

Il giorno martedì 20 dicembre 2016 17:02:48 UTC+1, Hartley Sweeten ha scritto:

On Tuesday, December 20, 2016 4:50 AM, Leonardo Romor wrote:
> Hello, regarding the implementation with grpc, it seems to be a difficult path to
> merge the async behavior with select() and the async engine of grpc (as pointed
> out here
> https://groups.google.com/forum/#!topic/grpc-io/9LzMrNUTKAM).
>
> So these are the choices imho
> a) we go multithread
> b) we use select with 0 timeout, (and asyncNext) but in this way we will not exploit anyhow the interrupts
> c) we instantiate multiple servers, a udp server for streaming position a tcp one for sending commands and another one to send pure gcode.
>
> do you have other options to add? 
> If not what do you suggest? I personally prefer the last one.

 

Well, option c implies option a. I think the only way to run multiple servers is
to go multithreaded.

Henner wrote this https://github.com/hzeller/threadless-server we could use something like this to avoid multithreading

 

Anyway... I like the idea of a udp server for streaming position as well as status
information. Commands, like starting/pausing/stopping/jogging, could also be
received through this server. The current tcp server is used as-is to receive
gcode. I don't think a separate "command" tcp server is needed.

 the second tcp is meant to be an interface that never blocks from which you can send commands.
The gcode server, as it is now, could always block (ie we send a huge gcode program and since we don't want to completely load it in memory, at a certain moment our tcp server will block).

 

Regards,
Hartley

[Henner Zeller]

On 20 December 2016 at 03:49, Leonardo Romor <leonard...@gmail.com> wrote:
> Hello, regarding the implementation with grpc, it seems to be a difficult
> path to merge the async behavior with select() and the async engine of grpc
> (as pointed out here
> https://groups.google.com/forum/#!topic/grpc-io/9LzMrNUTKAM).

Could grpc be fed with a simple file-descriptor and we can deal with
things around it ? In the end, it id just protocol buffers over some
socket, so maybe we can simulate that indpenednelty.

>
> So these are the choices imho
> a) we go multithread
> b) we use select with 0 timeout, (and asyncNext) but in this way we will not
> exploit anyhow the interrupts
> c) we instantiate multiple servers, a udp server for streaming position a
> tcp one for sending commands and another one to send pure gcode.

As Hartley said, (c) implies (a). which then means, that we have to do
some possibly annoying mutex locking of things.

We could multiplex servers with some event mechanism which multiplexes
using select() or poll(). That would also be able to deal with the
interrupt from the queue.

> do you have other options to add?
> If not what do you suggest? I personally prefer the last one.

In general, I'd try to avoid going with threads that meddle with the
same data - here we would have one thread putting stuff into the queue
and another thread figuring out what the current queue status and
position is while it needs to lock elements that are needed by both.
So thinking about strategies that makes sure that different threads
operate on different data with only minimal communication (message
queue or something) would be good. Given that we only have one core,
multiple threads don't necessarily buy anything.

I am not opposed to threads in general, but only if other options are
too nasty; so exploring the async stuff
I started an explorative branch "event-reading" for this
(https://github.com/hzeller/beagleg/tree/event-reading)
These are the first steps, hacked together quickly; Right now, it does
not do anything with the actual reading from the stream yet, but it
starts the loop in main() and operates the connecting server. And
TODOs in gcode-parser.h mentioning the next steps.

-h

>
> -l
>
>
> Il giorno venerdì 8 luglio 2016 13:16:40 UTC+2, Leonardo Romor ha scritto:
>>
>> Hi everyone,
>> this wants to be a continuation of the discussion started here.
>>
>> I think that the last Hartley post is a good starting point,
>>
>> we could use protocolbuffers or grpc to implement the interface.
>> I will list the stats that in my opinion would be important to show in the
>> form of a proto-like code, just to give an idea
>>
>> message BgStats {
>> string status; // A string or byte that gives the current status of the
>> machine, if it's paused, running, stopped, error(which error?) repeated
>> float pos = 3; // array containing the position in real world coordinates
>> for each axis
>> float temp; // the temperature thing
>> float spindle_speed;
>> repeated float pos; // Real world axes coordinates
>> repeated float speed; // speed? Or we could derive it outside beagleg
>> }
>>
>> and, a command request
>>
>> message BgCommand {
>> string command; // start, stop, pause
>> }
>>
>>
>> -lr
>

> --
> You received this message because you are subscribed to the Google Groups
> "beagleg-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to beagleg-dev...@googlegroups.com.
> To post to this group, send email to beagl...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/beagleg-dev/bb66f205-269f-4225-956f-9cce847898bb%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.

[Hartley Sweeten]

On Tuesday, December 20, 2016 2:06 PM, Henner Zeller wrote:
> On 20 December 2016 at 03:49, Leonardo Romor <leonard...@gmail.com> wrote:
>> Hello, regarding the implementation with grpc, it seems to be a difficult
>> path to merge the async behavior with select() and the async engine of grpc
>> (as pointed out here
>> https://groups.google.com/forum/#!topic/grpc-io/9LzMrNUTKAM).
>
> Could grpc be fed with a simple file-descriptor and we can deal with
> things around it ? In the end, it id just protocol buffers over some
> socket, so maybe we can simulate that indpenednelty.
>
>>
>> So these are the choices imho
>> a) we go multithread
>> b) we use select with 0 timeout, (and asyncNext) but in this way we will not
>> exploit anyhow the interrupts
>> c) we instantiate multiple servers, a udp server for streaming position a
>> tcp one for sending commands and another one to send pure gcode.
>
> As Hartley said, (c) implies (a). which then means, that we have to do
> some possibly annoying mutex locking of things.

If the udp socket is only returning "status" information we might be able
to get away without any mutex locking if the udp server treats erverything
as read-only. There is the possibility that information returned will not be
absolutely current but it should be close enough to real-time to be usable.

If the udp socket is eventually used for jog control (start, pause, etc.) and/or
jogging we will need mutex locking for any information that is modified. We
might be able to get away with a single lock that is requested then held by
some kind of protocol command received by the udp socket. This lock would
then be released later by another protocol command or after some timeout
without receiving any commands on the udp socket.

> We could multiplex servers with some event mechanism which multiplexes
> using select() or poll(). That would also be able to deal with the
> interrupt from the queue.
>
>> do you have other options to add?
>> If not what do you suggest? I personally prefer the last one.
>
> In general, I'd try to avoid going with threads that meddle with the
> same data - here we would have one thread putting stuff into the queue
> and another thread figuring out what the current queue status and
> position is while it needs to lock elements that are needed by both.
> So thinking about strategies that makes sure that different threads
> operate on different data with only minimal communication (message
> queue or something) would be good. Given that we only have one core,
> multiple threads don't necessarily buy anything.
>
> I am not opposed to threads in general, but only if other options are
> too nasty; so exploring the async stuff
> I started an explorative branch "event-reading" for this
> (https://github.com/hzeller/beagleg/tree/event-reading)
> These are the first steps, hacked together quickly; Right now, it does
> not do anything with the actual reading from the stream yet, but it
> starts the loop in main() and operates the connecting server. And
> TODOs in gcode-parser.h mentioning the next steps.

I have next week off. Hopefully I will get some time to look at this.

Hartley