my socket script does not receive all the data I send it - does anyone no why?
Kipkemoi
send it but not all? If I use a windows telnet application I able to
receive all the data.
My set up is that I have a piece of hardware that sends 1000 samples
(my hardware sends the sample plus carriage return char) of data
across the network and the script below listens for the data and
writes it to file. However I only ever manage to receive 841 samples
does anyone no why? I have played around with the buffer size but with
to no effect. I have removed the IP address.
set f [open "data.txt" w]
set s [socket some.where.com 30]
fconfigure $s -blocking 0 -buffersize 100
fileevent $s readable "readMe $s $f"
proc readMe {chan data} {
if {[gets $chan line] < 0} {
if {[eof $chan]} {
close $chan
close $outfile
puts "closing socket and file\n"
return
}
# Could not read a complete line this time; Tcl's
# internal buffering will hold the partial line for us
# until some more data is available over the socket.
} else {
puts $data "$line"
}
Gerald W. Lester
> Can anyone tell me why this script below receives most of the data I
> send it but not all? If I use a windows telnet application I able to
> receive all the data.
Are you using iocpsock (see http://wiki.tcl.tk/11304) -- if not, then you
need to and the problem will likely go away.
--
+------------------------------------------------------------------------+
| Gerald W. Lester |
|"The man who fights for his ideals is the man who is alive." - Cervantes|
+------------------------------------------------------------------------+
Alexandre Ferrieux
wrote:
> Can anyone tell me why this script below receives most of the data I
> send it but not all? If I use a windows telnet application I able to
> receive all the data.
>
> My set up is that I have a piece of hardware that sends 1000 samples
> (my hardware sends the sample plus carriage return char) of data
> across the network and the script below listens for the data and
> writes it to file. However I only ever manage to receive 841 samples
> does anyone no why? I have played around with the buffer size but with
> to no effect. I have removed the IP address.
Why are you using [gets] instead of [read] while your description
seems to indicate your protocol is binary ?
-Alex
David
I tried iocpsock and using the read command and got the exact same
result as before. If I increase the number of samples at the server
end e.g to 1800 samples then at the client end I received the first
1549 samples but nothing after. It is roughly the same percentage 86%
received as with the 1000 samples 84%.
My feeling is that its a problem related to the way I have set up
fconfigure. I can receive all the data by using a loop inside my proc
to repeatedly read the socket but this produced some duplication of
data and is obviously a messy way of doing things.
This is and example of the C code at the server end to give folks and
idea of the format I am sending the data in...I build up a buffer of
stings before sending it to the host pc
// get data point from memory, convert to a string and
store in tx buffer
tx_wr_pos_b += sprintf(tx_wr_pos_b,"%d\r", memory[x]);
...fill up the tx buffer before sending
// send data to host over simple socket
bytes_sent = send(conn->fd, tx_buf_b, tx_wr_pos_b -
tx_buf_b, 0);
Any more suggestions or hints?
Uwe Klein
> Any more suggestions or hints?
q: are you missing "real" samples or
do you just see CR/LF reduced CR ( or LF) ?
OK.
Take your original example.
remove the [fconfigure $sock -blocking 0]
remove the [fconfigure $sock -buffersize ...]
add
fconfigure $sock -buffering line
add a
puts stderr opts:[ fconfigure $sock]
to see how the socket is set up
change your fileevent proc to:
set ::cnt 0
proc 4fileevent {sock fd} {
if {[eof $sock]} {
# handle closedown
close $sock
flush $fd
close $fd
set ::closedown 1
}
incr ::cnt [ gets $sock linebuffer ]
puts $fd $linebuffer
}
vwait ::closedown
puts stderr "EOF: $::closedown ; bytes transfered:$::cnt"
uwe
Alexandre Ferrieux
>
> // send data to host over simple socket
> bytes_sent = send(conn->fd, tx_buf_b, tx_wr_pos_b -
> tx_buf_b, 0);
>
> Any more suggestions or hints?
Quite possibly your sender is the culprit. Instead of send() which is
rather fit for packet-based IO, prefer write() which is stream
oriented.
In any case, to avoid this guessing game, use tcpdump/wireshark to see
exactly what makes it to the wire. Also, please post your modified
code based on [read].
-Alex
tom.rmadilo
wrote:
I'm not seeing any binary indicators here. For that you need something
like this:
fconfigure $sockin -translation binary
Otherwise, it is very common to get the byte count wrong and maybe
terminate early, or sit around waiting for additional bytes.
But without knowing the protocol details (what are samples?)
impossible to say more. But this is just very characteristic behavior:
don't use -translation binary and data goes missing on the tcl end.
But use telnet and all the data arrives.
Alexandre Ferrieux
>
> fconfigure $sockin -translation binary
Oops you're right. I saw fconfigure, didn't even notice it lacked the
most important part :}
But since now it appears the content is rather textual (see the
sprintf's in the C code), it is a bit awkward to just report the
number of bytes, without saying how the text is distorted. This made
me assume a binary protocol.
> Otherwise, it is very common to get the byte count wrong and maybe
> terminate early, or sit around waiting for additional bytes.
Yes, in Windows there is that stupid "stop character", which could
very well account for brutal truncation.
In addition to verifying that [-translation binary] fixes the problem,
the OP could improve his understanding by looking at the kind of
distortion he observes (truncation, loss of 1 byte per CRLF, encoding
issues).
-Alex
David
problem turns out to be writing to file rather than with tcl sockets.
I installed "wireshark" as Alex suggested and it showed that all the
data was arriving at the host. Instead of writing to file I printed to
stdio and all the data appeared.
The solution that I have adopted is to append the data to a list and
then at the end write the whole list to file. I probably should have
been doing this from the start as it is a lot neater.
package require Iocpsock
set s [socket2 10.0.0.2 30]
fconfigure $s -blocking 0 -buffering line
fileevent $s readable "readMe $s"
set ::line_cnt 0
set ::new_data {}
proc readMe chan {
gets $chan linebuffer
if {$::line_cnt > 1798} {
# write data to file
set fd [open "ups_data.txt" w]
foreach line $::new_data {
puts $fd $line
}
puts $fd $linebuffer
# handle closedown
close $chan
close $fd
return
} else {
incr ::line_cnt 1
lappend ::new_data $linebuffer
}
}
As my old boss use to say "it the assumptions that get you".
Thank you again for everyone's help and time.
David