Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

late notification about socket unavailability - how to avoid loosing messages while sending

43 views
Skip to first unread message

tyz...@gmail.com

unread,
Jun 25, 2018, 5:27:38 AM6/25/18
to
Hello,

I'm developing TCL package for STOMP and during extensive tests I found problem with sending messages over the socket to the server - some messages get lost just before server disconnects.
I got notification about socket EOF after several 'puts'/'gets'/milliseconds

I'm using TCL 8.6.8 on Windows 7

Bellow you can find piece of code.
The idea is to redirect messages to file buffer when server disconnects.
Everything is working OK except I get lost some messages before TCL realized that socket does not exists anymore (in my case it can be hundreds messages).


My question is: do you have any solution for such problem? I just need solution to be properly/fast notify about server disconnect.

Thanks,
/TT

set fd [socket $ip $port]
fconfigure $fd-blocking 0 -eofchar {} -translation {auto lf} -encoding utf-8 -buffering none
fileevent $fd readable [list [self]::my HandleInput]

set fileFD [open c:/tmp/log w]
fconfigure $fileFD -eofchar {} -buffering none -encoding utf-8

method pub {data} {
#send data to file
puts $fileFD $data

#send data to server
if {[my send $data] < 0} {
#redirect data to buffer file
}

}

method send {data} {
....
if {[my SendTo $msg] < 0} {return -2}
my HandleInput
return 0
}

method SendTo {msg} {
set ret 0
if {[catch {puts $fd $msg} err]} {
my Error "Server not reachable"
catch {close $fd}
set ret -2
set isConnected -1
}
return $ret
}
method HandleInput {} {
set ret {}
if { [eof $fd]} {
my Error "Server suddenly disconnects before read"
catch {close $fd}
set isConnected -1
} else {
set d [my HandleReadChannel]
if {[fblocked $fd]} {
return
}
....
}
return $ret
}
method HandleReadChannel {{len -1}} {
set ret {}
if {$len < 0} {
set ret [gets $fd]
} else {
set ret [read $fd $len]
}
if {[eof $fd]} {
my Error "Server suddenly disconnects after read"
catch {close $fd}
set isConnected -1
return -level 2 {}
}
return $ret
}

Harald Oehlmann

unread,
Jun 25, 2018, 10:45:18 AM6/25/18
to
Hi TT,
IMHO if the write does not work, it should still be possible to read any
remaining bytes in the read buffer.

Do you know about half close ?
You signal close, but can still receive and read data ?

http://www.tcl.tk/man/tcl8.6/TclCmd/close.htm

close $h write

Harald

Brad Lanam

unread,
Jun 25, 2018, 5:09:50 PM6/25/18
to
On Monday, June 25, 2018 at 2:27:38 AM UTC-7, tyz...@gmail.com wrote:
> Hello,
>
> I'm developing TCL package for STOMP and during extensive tests I found problem with sending messages over the socket to the server - some messages get lost just before server disconnects.
> I got notification about socket EOF after several 'puts'/'gets'/milliseconds
>
> I'm using TCL 8.6.8 on Windows 7
>
> Bellow you can find piece of code.
> The idea is to redirect messages to file buffer when server disconnects.
> Everything is working OK except I get lost some messages before TCL realized that socket does not exists anymore (in my case it can be hundreds messages).
>
>
> My question is: do you have any solution for such problem? I just need solution to be properly/fast notify about server disconnect.


I expect you will need an application level acknowledgement.

There is tcp/ip buffering going on in your computer and in all the
routers and in the server, your sent messages could be anywhere
when the server stops receiving.

tyz...@gmail.com

unread,
Jun 27, 2018, 10:37:24 AM6/27/18
to
Harald,
The read is not a problem. I'm familiar with half close.

Brand,
I'm aware that the packages can be somewhere in between. Thus, I'm using VPN and local openVPN client to connect to server which allows me to effectively control connection.
Unfortunately, the app ACK per each message is not an option in this scenario - it would diametrical slowdown the communication (I need to send min 5 messages per 1ms).

I was experimenting with 'fileevent writable'. I want to 'lappend' buffer variable and when chanel becomes writable, just send the buffer. But I ended up with infinity loop - the 'writable' event is generated all the time.

I still have 2 additional ideas. Anyway, those are more workaround than solutions:
1. application send data to file while other thread 'tail'ing file sending data to server in packages every 10ms or similar.
2. similar to 1. but use list (RAM) instead of file.

Please, maybe someone have other solutions/thoughts.
All comments are very welcome.

Thanks,
/TT


Brad Lanam

unread,
Jun 27, 2018, 10:42:19 AM6/27/18
to
On Wednesday, June 27, 2018 at 7:37:24 AM UTC-7, tyz...@gmail.com wrote:
> On Monday, June 25, 2018 at 11:09:50 PM UTC+2, Brad Lanam wrote:
> > On Monday, June 25, 2018 at 2:27:38 AM UTC-7, tyz...@gmail.com wrote:
> > > Hello,
> > >
> > > I'm developing TCL package for STOMP and during extensive tests I found problem with sending messages over the socket to the server - some messages get lost just before server disconnects.
> > > I got notification about socket EOF after several 'puts'/'gets'/milliseconds
> > >
> > > I'm using TCL 8.6.8 on Windows 7
> > >
> > > Bellow you can find piece of code.
> > > The idea is to redirect messages to file buffer when server disconnects.
> > > Everything is working OK except I get lost some messages before TCL realized that socket does not exists anymore (in my case it can be hundreds messages).
> > >
> > >
> > > My question is: do you have any solution for such problem? I just need solution to be properly/fast notify about server disconnect.
> >
> >
> > I expect you will need an application level acknowledgement.
> >
> > There is tcp/ip buffering going on in your computer and in all the
> > routers and in the server, your sent messages could be anywhere
> > when the server stops receiving.
>
>
> Harald,
> The read is not a problem. I'm familiar with half close.
>
> Brand,
> I'm aware that the packages can be somewhere in between. Thus, I'm using VPN and local openVPN client to connect to server which allows me to effectively control connection.

This just adds more latency, not control.

> Unfortunately, the app ACK per each message is not an option in this scenario - it would diametrical slowdown the communication (I need to send min 5 messages per 1ms).

That's a design issue.
If you send/wait/send/wait, yes it will be slow.
There are other design alternatives.

Harald Oehlmann

unread,
Jun 27, 2018, 1:28:35 PM6/27/18
to
Am 27.06.2018 um 16:37 schrieb tyz...@gmail.com:
> Harald,
> The read is not a problem. I'm familiar with half close.

Ok, great.

> I was experimenting with 'fileevent writable'. I want to 'lappend' buffer variable and when chanel becomes writable, just send the buffer. But I ended up with infinity loop - the 'writable' event is generated all the time.

Then switch it off after it fired... A connected socket is nearly always
writable...

If speed is an issue, what about udp ?



0 new messages