How to write 100MB string to STDOUT

137 views
Skip to first unread message

Const V

unread,
May 8, 2022, 5:24:16 PM5/8/22
to golang-nuts
I'm submitting as a separate post to focus on the following problem I have:

writing 100MB string to STDOUT
w io.Writer
w.Write([]byte(s))

For smaller size string it is working. For bigger
like 100MB it is not.

I'm looking to submit chunks of the string like 1KB subsequently to STDOUT
until all string content is sent.

Amnon

unread,
May 8, 2022, 6:24:06 PM5/8/22
to golang-nuts
How did you reach the conclusion that it is not working?
What is your stdout connected to?

Const V

unread,
May 8, 2022, 6:45:15 PM5/8/22
to golang-nuts
In order to test I temporarily use a pipe in order to get the string from the stdout.

tempStdout := os.Stdout
r, w, _ := os.Pipe()
os.Stdout = w

r1 := bufio.NewReader(file)
s := ReadWithReadLine(r1)

// output to stdout is intercepted in order to get it as a string for testing
InputProcessing(strings.NewReader(s), os.Stdout)
 w.Close()

out, _ := ioutil.ReadAll(r)
os.Stdout = tempStdout

observed := string(out)

later - testing expected and observed outcome
....

if observed == expected {
t.Fatalf("\n%s, \nwant: %s\n", observed, expected)
}

Amnon

unread,
May 9, 2022, 12:12:02 AM5/9/22
to golang-nuts
Why not upload your entire unit test to https://go.dev/play/ so we can have a look at what you are doing?

Amnon

unread,
May 9, 2022, 12:16:55 AM5/9/22
to golang-nuts

From what you did post, the InputProcessing()
line would deadlock, because it is writing a lot of data to tempStdout, while noone is reading it.

I would instead do something like

out := new(bytes.Buffer)
InputProcessing(strings.NewReader(s), out)
observed := out.String()

Reply all
Reply to author
Forward
Message has been deleted
0 new messages