io.CopyN occasionally fails with unexpected EOF on large ranged reads

[Kanak Bhatia]

Hi all,

I'm encountering an intermittent issue while using io.CopyN to copy 129MB object from reader to writer.

The problem:
On some iterations, io.CopyN(dst, src, n) fails with unexpected EOF even though the same logic succeeds in most runs. The failure typically happens after 125–130 MB and returns ~1MB less data than expected.

type debugReader struct {
    r          io.Reader
    totalBytes int64
    nextLogMB  int64
}

func (d *debugReader) Read(p []byte) (int, error) {
    n, err := d.r.Read(p)
    d.totalBytes += int64(n)
    if d.totalBytes >= 125*1024*1024 {
        currentMB := d.totalBytes / (1024 * 1024)
        if currentMB >= d.nextLogMB {
            fmt.Printf("🔵 Read %d MB so far\n", currentMB)
            d.nextLogMB = currentMB + 1
        }
    }
    return n, err
}

type debugWriter struct {
    w          io.Writer
    totalBytes int64
    nextLogMB  int64
}

func (d *debugWriter) Write(p []byte) (int, error) {
    n, err := d.w.Write(p)
    d.totalBytes += int64(n)
    if d.totalBytes >= 125*1024*1024 {
        currentMB := d.totalBytes / (1024 * 1024)
        if currentMB >= d.nextLogMB {
            fmt.Printf("🟡 Written %d MB so far\n", currentMB)
            d.nextLogMB = currentMB + 1
        }
    }
    return n, err
}

func SafeCopyN(dst *debugWriter, src *debugReader, n int64) (int64, error) {
    fmt.Printf("\n🔁 Starting SafeCopyN for %d bytes\n", n)
    written, err := io.CopyN(dst, src, n)
    fmt.Printf("📊 Total bytes read:    %d\n", src.totalBytes)
    fmt.Printf("📊 Total bytes written: %d\n", dst.totalBytes)

    const allowedDrift = 128 * 1024

    if err != nil {
        diff := n - written
        if err == io.ErrUnexpectedEOF && diff <= allowedDrift {
            fmt.Printf("⚠  Accepting unexpected EOF: copied %d of %d bytes (drift: %d bytes)\n", written, n, diff)
            return written, nil
        }

        fmt.Printf("❌ CopyN failed: wrote %d of %d bytes, err: %v\n", written, n, err)
        if src.totalBytes != dst.totalBytes {
            fmt.Printf("📉 Mismatch: %d bytes read but not written\n", src.totalBytes-dst.totalBytes)
        }
    } else {
        fmt.Printf("✅ Successfully copied %d bytes\n", written)
    }

    return written, err
}


---

Here's a real log from one of the failed runs (test runs 20 times, fails 2):


🔁 Starting SafeCopyN for 135264256 bytes
🔵 Read 125 MB so far
🟡 Written 125 MB so far
🔵 Read 126 MB so far
🟡 Written 126 MB so far
🔵 Read 127 MB so far
🟡 Written 127 MB so far
📊 Total bytes read:    134215680
📊 Total bytes written: 134215680
❌ CopyN failed: wrote 134215680 of 135264256 bytes, err: unexpected EOF
📉 Mismatch: 0 bytes read but not written

Notes:

This only happens 1 to 2 times out of 20 iteration.

No server-side encryption involved.
---

Question:

Has anyone seen something similar? Could this be:

Timeout/internal truncation in io.CopyN?

Should io.CopyN tolerate io.ErrUnexpectedEOF more gracefully in this case?


Any insights or guidance appreciated!

[Steven Hartland]

What's the source and target, are they networked if so likely a network issue.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/golang-nuts/56d2219e-32ab-4950-ab49-83b5fbbfc4fen%40googlegroups.com.

[Kanak Bhatia]

Its not networked

// Read the data back
fmt.Println("Fetching object back using GetObject...")
r, err := c.GetObject(context.Background(), bucketName, objectName, minio.GetObjectOptions{})
if err != nil {
    logError(testName, function, args, startTime, "", "GetObject failed", err)
    return
}
defer r.Close()
fmt.Println("GetObject successful")


// Stat the object
st, err := r.Stat()
if err != nil {
logError(testName, function, args, startTime, "", "Stat object failed", err)
return
}
fmt.Printf("Stat object successful. Object size: %d bytes\n", st.Size)

if st.Size != int64(bufSize) { // bufSize is original object size (129 MB)
logError(testName, function, args, startTime, "", fmt.Sprintf("Number of bytes does not match, expected %d, got %d", bufSize, st.Size), err)
return
}
fmt.Println("Object size verified against expected buffer size")


--------------------------------------------------------

// Comparison function after seek
cmpData := func(r io.Reader, start, end int) {
    if end-start == 0 {
        fmt.Printf("cmpData: no bytes to compare (start: %d, end: %d)\n", start, end)
        return
    }
    fmt.Printf("cmpData: comparing bytes from %d to %d...\n", start, end)
    buffer := bytes.NewBuffer([]byte{})
    // This is the critical line where the error occurs
    if _, err := io.CopyN(buffer, r, int64(bufSize)); err != nil { 
        if err != io.EOF {
            logError(testName, function, args, startTime, "", "CopyN failed", err)
            return
        }
        // ... rest of cmpData function ...
    }
    if !bytes.Equal(buf[start:end], buffer.Bytes()) {
        logError(testName, function, args, startTime, "", "Incorrect read bytes v/s original buffer", err)
        return
    }
    fmt.Println("cmpData: byte comparison successful — data matches original buffer")
}


----------------------------------------------------------------------------

[Robert Engels]



I suspect you maybe have an anti virus software (or something like it) that is messing up by reading/interrupting the process. What OS?

On Jul 7, 2025, at 1:14 PM, Kanak Bhatia <kanakb...@gmail.com> wrote:



To view this discussion visit https://groups.google.com/d/msgid/golang-nuts/CABXaED640tj-JF%2Bss0pH72WPxXR%3D4O0tRWE98XXzieHDCRbf_A%40mail.gmail.com.

[Kanak Bhatia]

Using windows 11 + wsl 

[Kanak Bhatia]

I meant windows 11, but i am using wsl internally for everything

[Kanak Bhatia]

After getting object a validation is there to check if returned object size is same as put object. (r.Stat() function shown in above code.)

After that only copyN function is used.

On Tue, 8 Jul 2025, 00:00 Steven Hartland, <stevenm...@gmail.com> wrote:

You said not network but your example is using minio which I believe is a S3 compatible storage which sounds very much like network based storage?

[Robert Engels]

I would try your program in a Linux vm. I highly suspect an OS issue. If it works, I would try on a pure Windows 10 VM. 

On Jul 7, 2025, at 1:25 PM, Kanak Bhatia <kanakb...@gmail.com> wrote:



[Kanak Bhatia]

Thanks. 

If required I can share more details if you want.

[Robert Engels]

It is probably beneficial to publish a self contained test to github that people can run on various OSs. If it’s all local files it should be easy enough for you to create. 

On Jul 7, 2025, at 2:13 PM, Kanak Bhatia <kanakb...@gmail.com> wrote:



[Robert Engels]

One place I have seen stuff like this happens is with USB drives with cheap/faulty controllers - especially with memory stick type devices. 

On Jul 7, 2025, at 2:31 PM, Robert Engels <ren...@ix.netcom.com> wrote:



[Brian Candler]

On Monday, 7 July 2025 at 19:14:49 UTC+1 Kanak Bhatia wrote:

Its not networked

// Read the data back
fmt.Println("Fetching object back using GetObject...")
r, err := c.GetObject(context.Background(), bucketName, objectName, minio.GetObjectOptions{})

minio seems very networked to me. Can you make a standalone Go program which reproduces the issue using local filesystem only? Preferably with minimal third-party dependencies?

The process of trimming down your code to achieve this may in itself help to narrow down where the problem lies.

[Kanak Bhatia]

Sure. Will share it.

--

You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/golang-nuts/bd8cdc7b-9e7b-4864-b5d2-39d9af51d812n%40googlegroups.com.

[Kanak Bhatia]

Hi All, 

You were right. There is underlying Minio Sever side code issue which breaks ths connection sometimes of reader used in copyN resulting in unexpectedEOF.