Reading the serial port line by line

[Foad S Farimani]

Hello guys,

I am very novice to tcl language and I am trying to solve this issue:

https://stackoverflow.com/questions/54934867/replicate-arduinos-serial-monitor-on-scilab-consol-to-read-a-pair-of-comma-sepe

Where I want to read a series of numbers in CSV format. I tried solving it myself as I have explained here:

https://stackoverflow.com/questions/54935990/read-the-latest-line-of-a-serial-port

however I have not succeed so far and the suggestions I have got on the SO are also not working. Basically I want to listen to the serial port and then export the last nonempty line in the format of CSV :

120, 300, 400

into a string variable. I would appreciate if you could help me with this.

P.S. you can use SimulIDE and com0com as I have explained here:

https://sourceforge.net/p/simulide/discussion/general/thread/a53c097f6a/#34c6/7d5f/a80d

if you don't have the hardware.

[Rich]

Foad S Farimani <f.s.fa...@gmail.com> wrote:
> Hello guys,

>
> Where I want to read a series of numbers in CSV format. I tried solving it myself as I have explained here:
>
> https://stackoverflow.com/questions/54935990/read-the-latest-line-of-a-serial-port

Your solution page shows you were given the correct answer. But your
reply simply says "still does not work" (without revealing *anything*
about what change you made).

How do you expect others to read your mind, remotely sense what changes
you made to your code without ever seeing it, and answer your question
when you hide all the information from them?

And the answer you were given is correct. If you want to read a
channel (does not matter that it is a serial port) line by line, you
should use the "gets" command to do that.

The "read" command does not do line-by-line itself (you would have to
further wrap line by line handling on top of the data read returns.
But why do that when "gets" already includes all that "line-by-line"
handling internally?

[Robert Heller]

The only possible "gotcha" is line termination: Tcl *should* do this
automagically, but if the OP hardwires one scheme in the Tcl code and the
uses another in the Arduino code, things won't work.

The other issue is if the OP is using a GUI and is NOT using fileevent
(properly!)...

Yes, we do need more info from the OP. I have written a Tcl/Tk programs that
talk to Arduino and Arduino-like MCUs, so this is an easy problem.

>
>
>

--
Robert Heller -- 978-544-6933
Deepwoods Software -- Custom Software Services
http://www.deepsoft.com/ -- Linux Administration Services
hel...@deepsoft.com -- Webhosting Services

[Foad S Farimani]

On Saturday, March 2, 2019 at 3:50:36 PM UTC+1, Rich wrote:
> Foad S Farimani <f.s.fa...@gmail.com> wrote:
> > Hello guys,
> >
> > Where I want to read a series of numbers in CSV format. I tried solving it myself as I have explained here:
> >
> > https://stackoverflow.com/questions/54935990/read-the-latest-line-of-a-serial-port
>
> Your solution page shows you were given the correct answer. But your
> reply simply says "still does not work" (without revealing *anything*
> about what change you made).
>
> How do you expect others to read your mind, remotely sense what changes
> you made to your code without ever seeing it, and answer your question
> when you hide all the information from them?

Well, I had left a comment in that page explaining why that solution does not work. But in short the issue was that the function returned empty or incomplete lines. I found a solution myself which you may see here:

https://stackoverflow.com/a/54960839/4999991

comments to improve the solution are highly appreciated.

>
> And the answer you were given is correct. If you want to read a
> channel (does not matter that it is a serial port) line by line, you
> should use the "gets" command to do that.
>
> The "read" command does not do line-by-line itself (you would have to
> further wrap line by line handling on top of the data read returns.
> But why do that when "gets" already includes all that "line-by-line"
> handling internally?

i actually ended up using the gets command except I had to also check that the queue is not empty.

[Foad S Farimani]

On Saturday, March 2, 2019 at 5:28:09 PM UTC+1, Robert Heller wrote:
> At Sat, 2 Mar 2019 14:50:33 -0000 (UTC) Rich <ri...@example.invalid> wrote:
>
> >
> > Foad S Farimani <f.s.fa...@gmail.com> wrote:
> > > Hello guys,
> > >
> > > Where I want to read a series of numbers in CSV format. I tried solving it myself as I have explained here:
> > >
> > > https://stackoverflow.com/questions/54935990/read-the-latest-line-of-a-serial-port
> >
> > Your solution page shows you were given the correct answer. But your
> > reply simply says "still does not work" (without revealing *anything*
> > about what change you made).
> >
> > How do you expect others to read your mind, remotely sense what changes
> > you made to your code without ever seeing it, and answer your question
> > when you hide all the information from them?
> >
> > And the answer you were given is correct. If you want to read a
> > channel (does not matter that it is a serial port) line by line, you
> > should use the "gets" command to do that.
> >
> > The "read" command does not do line-by-line itself (you would have to
> > further wrap line by line handling on top of the data read returns.
> > But why do that when "gets" already includes all that "line-by-line"
> > handling internally?
>
> The only possible "gotcha" is line termination: Tcl *should* do this
> automagically, but if the OP hardwires one scheme in the Tcl code and the
> uses another in the Arduino code, things won't work.

I'm not sure if I understand what you mean here. I actually used SimulIDe and com0com to generate the serial data. you may find more information here:

https://sourceforge.net/p/simulide/discussion/general/thread/a53c097f6a/#34c6/7d5f/a80d

>
> The other issue is if the OP is using a GUI and is NOT using fileevent
> (properly!)...

Would you please elaborate on why fileevent and maybe give some examples? can it be used to improve my solution here:

https://stackoverflow.com/a/54960839/4999991

>
> Yes, we do need more info from the OP. I have written a Tcl/Tk programs that
> talk to Arduino and Arduino-like MCUs, so this is an easy problem.

It would be great if you could provide more info. Maybe upload the codes on a Github Gist?

[Foad S Farimani]

A full demonstration and instructions here on Reddit:

https://www.reddit.com/r/electronics/comments/awkoka/creating_a_virtual_arduino_in_simulide_and_serial/

[Robert Heller]

sourceforge.net is not a dialup friendly website, and I have a dialup
connection.

>
>
> >
> > The other issue is if the OP is using a GUI and is NOT using fileevent
> > (properly!)...
>
> Would you please elaborate on why fileevent and maybe give some examples? can it be used to improve my solution here:
>
> https://stackoverflow.com/a/54960839/4999991
>
> >
> > Yes, we do need more info from the OP. I have written a Tcl/Tk programs that
> > talk to Arduino and Arduino-like MCUs, so this is an easy problem.
>
> It would be great if you could provide more info. Maybe upload the codes on a Github Gist?

https://github.com/RobertPHeller/ReflowToasterOven2

Look at "ToasterConsole.tcl".

[Foad S Farimani]

> https://github.com/RobertPHeller/ReflowToasterOven2
> Look at "ToasterConsole.tcl".

Looking at

https://github.com/RobertPHeller/ReflowToasterOven2/blob/master/ToasterConsole.tcl

and here

https://www.tcl.tk/man/tcl8.4/TclCmd/fileevent.htm


Should it be like:

proc GetData {chan} {
if {![eof $chan]} {
puts [gets $chan]
} else {
catch {close $chan}
set done 1
return
}
}

set fp [open $port r]

fconfigure $fp -mode 9600,n,8,1 -blocking no -buffering none -handshake none -translation {crlf crlf}

fileevent $chan readable [list GetData $fp]

[Robert Heller]

Try this:

proc GetData {chan} {
if {[gets $chan line] < 0} {

catch {close $chan}
set ::done 1

} else {
puts $line
}
}

The eof command does not work *before* a read. It is more reliable to use
gets result when passing a var to capture the input. In that form gets
returns the length of the line or -1 on error/eof. Also setting a local
variable is useless. I presumed that you wanted to set a global variable, in
which case putting it in the global namespace is needed.

Oh, and the {crlf crlf} for translation *presumes* that your Serial is
configured to send both CR (\r) and LF (\n) and end of line.

[Foad S Farimani]

this worked finally:

for {set i 0} {$i < 40} {incr i} {
set line ""
while {$line == ""} {
fileevent $fp readable [gets $fp line]
}
puts $line
}

[Robert Heller]

WHAT? The above makes no sense. You don't put fileevent in a loop -- there
is no point. You are setting the fileevent script to be whatever the Arduino
is sending.

This is what you want. I expect you don't actually need fileevent. You only
need it if you need to do the I/O asyncroniously or as a background behind a
GUI or something like that.

for {set i 0} {$i < 40} {incr i} {
set line ""
while {$line == ""} {

gets $fp line
}
puts $line
}

[Foad S Farimani]

It eventually needs to be in a GUI. But I'm not at that point yet. I guess I still need to learn about fileevent. Thanks for your help anyways.

[Rich]

Foad S Farimani <f.s.fa...@gmail.com> wrote:

Fileevent does what its name sounds like. It schedules a script to be
executed at a later time when an 'event' occurs on a file. In the case
of fileevent, the 'events' are data available to be read, or space in
an output buffer into which to write data.

But you don't call it in a loop (as your posting showed). You call it
once to connect the file descriptor to the script (the script is
usually a call to a proc) and then enter the event loop. The calls to
your script/proc occur later, when data is actually available to be
read.

[Foad S Farimani]

Thanks guys for your great support. Obviously my knowledge of Tcl and programing in general is very primitive. It would be great if any of you guys could contribute to the original SciLab Serial Communication Toolbox (SCT):

https://github.com/sengupta/Scilab-Serial

SciLab is a great MATLAB like platform (both GUI and syntax) and it also has scicos / xcos which aims to be a SIMULINK replica. The main issue is the lack of support as I hav ementioned here:

https://www.reddit.com/r/scilab/comments/atf5vq/where_is_the_best_place_for_scilab_xcos_scicoslab/

Ideally the SCT should replicate MATLAB's functionality:

https://www.mathworks.com/help/matlab/ref/serial.html

The main issue is the lack of classes-objects in SciLab. I see that newer versions of Tcl have OOP. so maybe a serial class could be made in Tcl and then using TCL_EvalStr / TCL_EvalFile ...

[Ralf Fassel]

* Foad S Farimani <f.s.fa...@gmail.com>

| I found a solution myself which you may see here:
>
| https://stackoverflow.com/a/54960839/4999991
>
| comments to improve the solution are highly appreciated.

I can only speak for myself: I'm reading comp.lang.tcl with a purpose,
that purpose being to read and eventually help on comp.lang.tcl, not on
Github, not on Sourceforge, not on SO.

So if you're seeking advice/help on c.l.t., I would suggest you put all
relevant information *here*, and not force those willing to help on a
multitude of different WEB sites. (Plus, that link above has not one bit
of plain TCL code in it...)

Yes, posting relevant content *here* requires a bit of effort on your
side. I myself often found that in the process of preparing a small
example for the question at hand I often found the answer to the
problems, and if not the answer, at least more knowledge in TCL.

My EUR 0.01.
R'

[Gunnar Schmidt]

Hello,

thank you all for this input. I m struggling, that my buffer is getting very large. I try to figure out, how to add to each line in the buffer a timestamp.

Is it possible to find out, when the message is send by the device via serial communication.

I used your code:

function buf = readserialline(h)
tmpbuf = emptystr();
while tmpbuf == emptystr()
//TCL_EvalStr("gets " + h + " ttybuf"); Todo: Had some issues, if queue of serialstatus == 75
//https://stackoverflow.com/questions/54935990/read-the-latest-line-of-a-serial-port
// Todo: read last line of buffer and add timestamp
TCL_EvalStr("binary scan [gets "+h+"] cu* ttybuf");
//tmpbuf = TCL_GetVar("ttybuf");
tmpbuf = ascii(evstr(TCL_GetVar("ttybuf")));
end
buf = tmpbuf;

endfunction

[Gunnar Schmidt]

How can you generate a time stamp whenever the serial port API generates an event to indicate that data was received as well as after newline and/or carriage return characters?

[Rich]

Gunnar Schmidt <gunnar....@moveit.rocks> wrote:
> How can you generate a time stamp whenever the serial port API
> generates an event to indicate that data was received

In your event handler, use the [clock] command to generate a timestamp
in your desired format.

> as well as after newline and/or carriage return characters?

For that you will have to monitor the incoming data for
newline/carriage return characters and when you find one, use [clock]
to generate a timestamp.