fileevent writable on socket

[EL]

Hello,

following up on my previous question and the [fconfigure -blocking 0]
issue with sockets.

I did now rewrite my code to make it more bullet proof and also try to
use [fileevent writable]. The relevant parts look like the code below.
Essentially I set up a writable event, which consumes every message in a
_Queue (a list variable) and puts it on the socket, when the socket is
writable. Sending to the socket does only [lappend] the message on the
_Queue, from which it is consumed in the writable event.
Now I recognized that this event is fired very often, which hurts some
other parts of my application. Also I read in the [puts] manual that
[puts] does apparently keep an internal buffer and puts the message on
the underlying socket just as soon as this is possible - the last
section starting with "When the output buffer fills up..." here:
http://tcl.tk/man/tcl8.6/TclCmd/puts.htm

So my question is: is it at all necessary to self implement a message
queue and consume it via a [fileevent writable]? Can strange things
happen in border conditions, if I leave that part out and do [puts $sock
...] directly?
And if it is not necessary, what is [fileevent writable] good for at
all, except checking if the socket is connected, when in -async mode?


proc connect {port} {
global _Connected
set sock [socket -async localhost $port]
fconfigure $Socket -translation auto -buffering line -blocking 0
fileevent $sock writable [list connectionEstablished $sock]
vwait _Connected
if {$_Connected ne "success"} {
throw {MY ERR} "connection cannot be established: $_Connected"
}
unset _Connected
}

proc connectionEstablished {sock} {
global _Connected
fileevent $sock writable {}
set err [fconfigure $sock -error]
if {$err != {}} {
set _Connected $err
return
}
fileevent $sock readable [list handleRead $sock]
fileevent $sock writable [list handleWrite $sock]
set _Connected success
}

proc handleWrite {sock} {
global _Queue
while {[info exists _Queue] && ![fblocked $sock] && $_Queue != {}} {
set msg [lindex $_Queue 0]
puts $sock $msg
set _Queue [lrange $_Queue 1 end]
}
}

proc handleRead {sock} {
...
}

proc send {sock msg} {
global _Queue
lappend _Queue $msg
}

set s [connect 1234]
send $s "Hello World"

--
EL

[Ralf Fassel]

* EL <e...@noreply.spam>

| So my question is: is it at all necessary to self implement a message
| queue and consume it via a [fileevent writable]?

Most of the times not, since TCL does all the buffering an flushing for
you:

man 'puts':

When the output buffer fills up, the puts command will normally
block until all the buffered data has been accepted for output by
the operating system. If channelId is in nonblocking mode then
the puts command will not block even if the operating system
cannot accept the data. Instead, Tcl continues to buffer the
data and writes it in the background as fast as the underlying
file or device can accept it. The application must use the Tcl
event loop for nonblocking output to work; otherwise Tcl never
finds out that the file or device is ready for more output data.

It is possible for an arbitrarily large amount of data to be
buffered for a channel in nonblocking mode, which could consume a
large amount of memory. To avoid wasting memory, non- blocking
I/O should normally be used in an event-driven fashion with the
fileevent command (do not invoke puts unless you have recently
been notified via a file event that the channel is ready for more
output data).

In general the code you've shown tries to implement the latter part, but
as you had observed, the fileevent firing repeatedly when there is
nothing to write has a performance impact on the application (same as
e.g. read-fileevents on a regular file which will fire permanently).

So I'd either not use fileevent writeable at all, or only enable it when
there is actually data to write (i.e. the Queue is non-empty), or the
internal TCL buffers as observed by "chan pending" become very full.

HTH
R'

[EL]

On 27.07.2021 12:36, Ralf Fassel wrote:
> or only enable it when
> there is actually data to write

That seems logical, I think I'll go with this:

proc send {sock msg} {
global _Queue
lappend _Queue $msg

fileevent $sock writable [list handleWrite $sock]
}

proc handleWrite {sock} {
global _Queue

while {[info exists $_Queue] && $_Queue != {}} {
puts $sock [lindex $_Queue 0]

set _Queue [lrange $_Queue 1 end]
}

fileevent $sock writable {}
}


Thanks for the hint :)


--
EL

[Ralf Fassel]

* EL <e...@noreply.spam>

| proc handleWrite {sock} {
| global _Queue
| while {[info exists $_Queue] && $_Queue != {}} {
| puts $sock [lindex $_Queue 0]
| set _Queue [lrange $_Queue 1 end]
| }
| fileevent $sock writable {}
| }

There are some quirks here:

- [info exists $_Queue] will probably never return true
I think you mean
[info exists _Queue]
here

- If you dump the complete contents of the queue to the socket, there is
no point in the fileevent at all.

Dump one item, and let the fileevent trigger again if there are more
items in the queue:

proc handleWrite {sock} {
global _Queue

if {[info exists _Queue] && [llength $_Queue] > 0} {

puts $sock [lindex $_Queue 0]
set _Queue [lrange $_Queue 1 end]
}

if {![info exists _Queue] || [llength $_Queue] == 0} {
fileevent $sock writable {}
}
}

HTH
R'

[EL]

On 27.07.2021 21:41, Ralf Fassel wrote:
> There are some quirks here:
>
> - [info exists $_Queue] will probably never return true
> I think you mean
> [info exists _Queue]
> here

Ah sure. In reality the _Queue is a TclOO member variable and it always
exists. I just made that up for simplicity and accidentally introduced a
typo ;).

> - If you dump the complete contents of the queue to the socket, there is
> no point in the fileevent at all.
>
> Dump one item, and let the fileevent trigger again if there are more
> items in the queue:
>
> proc handleWrite {sock} {
> global _Queue
> if {[info exists _Queue] && [llength $_Queue] > 0} {
> puts $sock [lindex $_Queue 0]
> set _Queue [lrange $_Queue 1 end]
> }
> if {![info exists _Queue] || [llength $_Queue] == 0} {
> fileevent $sock writable {}
> }
> }

Ok, yes that makes sense. Thanks a lot!


--
EL