Should net.Conn transform function retain the original data segmentation?
Xiaoyi Shi
Hi all,
To clarify, a net.Conn transform function transforms data write to/read from a connection, an example is as follow:
type TransformedConn struct {
net.Conn
}
func (c *TransformedConn) Read(p []byte) (int, error) {
n, err := c.Conn.Read(p)
transform(p)
return n, err
}
func (c *TransformedConn) Write(p []byte) (int, error) {
t := append([]byte{}, p...) // as `p` must not be modified per io.Writer doc
transform(t)
return c.Conn.Write(t)
}Similarly, crypto/cipher.StreamReader and crypto/cipher.StreamWriter perform a similar operation.
As required by io.Writer.Write(), it cannot modify the input buffer. In case the size of the buffer is huge, creating a buffer of the same size will be very expensive. So I'm wondering if it's ok for such transform function to write data out in smaller chunks, to allow us using a smaller scratch space?
I've found such chunking technique will cause some program to fail, especially the ones use io.ReadFull() and io.ReadAtLeast(). Because they implicitly expect that the data to arrive will from a single read() call.
Could someone point me a doc or a spec on how to write such transform functions?
Thanks!
Dave Cheney
James Bardin
James Bardin
Though in this case I assume you must be using a TCP connection, so there is no concept of a "message" and hence to direct connection between the write size and the read size. If something other than UDP is expecting the full message in a single Read call, it is incorrectly written and you should file a bug.