i/o timeout when using bufio on net connection (tcp)

[Vasiliy Tolstov]

Hi! I have some client-server app, that uses tcp to connect to remote server.
After connection established i'm try to download/upload 21G file
(download/upload in the same time so after recieve 26K data i'm send
it to remote server) after 700-900Mb i have i/o timeout errors:
dst write tcp [2a04:ac00:6:1::1]:57048->[2a04:ac00:4:10:ec4:7aff:fe54:cb0a]:16509:
i/o timeout

tcpdump on server says that when i have this errors no traffic goes
from my to remote server.

Data flow not simple, but main parts are in
https://github.com/vtolstov/go-libvirt/blob/master/client/stream.go#L141
and
https://github.com/vtolstov/go-libvirt/blob/master/client/rpc.go#L246

what can i do to fix this timeouts ?

--
Vasiliy Tolstov,
e-mail: v.to...@selfip.ru

[Kevin Johnson]

Hi Vasiliy,

I spent a few minutes looking at your program and a couple things jumped out at me.  As you are probably aware, writes on a network socket usually return the number of bytes written,  This is because if the TCP buffer fills, then the write call will return only the amount that got added.  I noticed you used a timeout on the TCP conn, which probably puts the connection into non-blocking mode.  I would be surprised if binary.write was smart enough to attempt multiple writes in case of a partial write.  You might try removing the time-out and see if that has any effect.  (Network non-blocking writes may wait for the entire write to succeed before returning).  You might also use the binary.write to write to a local block of bytes and then use the conn send to send them manually, resending the parts when the writes only partially complete.

Best of luck,

Kevin Johnson

[Vasiliy Tolstov]

2017-04-17 19:05 GMT+03:00 Kevin Johnson <jkevi...@gmail.com>:
> Hi Vasiliy,
>
> I spent a few minutes looking at your program and a couple things jumped out
> at me. As you are probably aware, writes on a network socket usually return
> the number of bytes written, This is because if the TCP buffer fills, then
> the write call will return only the amount that got added. I noticed you
> used a timeout on the TCP conn, which probably puts the connection into
> non-blocking mode. I would be surprised if binary.write was smart enough to
> attempt multiple writes in case of a partial write. You might try removing
> the time-out and see if that has any effect. (Network non-blocking writes
> may wait for the entire write to succeed before returning). You might also
> use the binary.write to write to a local block of bytes and then use the
> conn send to send them manually, resending the parts when the writes only
> partially complete.

Ok, thanks! I'm try to write to buffer and use simple net.Conn Write.