Problem reading TCP Stream using readbytes (works with array comprehension)

[Olav Andreas Lindekleiv]

Hi.

I recently discovered Julia, and like it a lot so far. After porting some code I'd written in Go, I decided to write a simple SCGI library to be run behind Nginx for a toy project.

The code can be found here: https://github.com/oal/SCGI.jl

I use readbytes to read the incoming request headers from Nginx, which works as expected. Though when I try to read the request body, it blocks (waits for more data until the request times out) even though I tell it to only read the length of the request body (CONTENT_LENGTH). However, when I use array comprehension and do "ns.content = [read(ns.socket, Uint8) for i = 1:int(ns.headers["CONTENT_LENGTH"])]", it reads what I want, and returns the response.

Here's the line that works. The line below (commented out) will block, even though it should read the same amount of bytes:
https://github.com/oal/SCGI.jl/blob/85117bb3fabed3e88b305b16eca066046c76ec33/scgi.jl#L53

Or am I missing something?

Julia version: 0.2.0-prerelease+3051

[Steven G. Johnson]

On Tuesday, August 13, 2013 10:44:35 AM UTC-4, Olav Andreas Lindekleiv wrote:

I use readbytes to read the incoming request headers from Nginx, which works as expected. Though when I try to read the request body, it blocks (waits for more data until the request times out) even though I tell it to only read the length of the request body (CONTENT_LENGTH). However, when I use array comprehension and do "ns.content = [read(ns.socket, Uint8) for i = 1:int(ns.headers["CONTENT_LENGTH"])]", it reads what I want, and returns the response.

That's weird, the readbytes function should be calling the one in io.jl, which calls readbytes! in io.jl, which just calls read(stream, Uint8) the requested # of times, so it should be no different than the list comprehension.   Can you try inserting a println or two into io.jl's readbytes!(s::IO, ....) function (not the IOStream version) to see where it is blocking?

[Steven G. Johnson]

On Tuesday, August 13, 2013 11:08:57 PM UTC-4, Steven G. Johnson wrote:

On Tuesday, August 13, 2013 10:44:35 AM UTC-4, Olav Andreas Lindekleiv wrote:

I use readbytes to read the incoming request headers from Nginx, which works as expected. Though when I try to read the request body, it blocks (waits for more data until the request times out) even though I tell it to only read the length of the request body (CONTENT_LENGTH). However, when I use array comprehension and do "ns.content = [read(ns.socket, Uint8) for i = 1:int(ns.headers["CONTENT_LENGTH"])]", it reads what I want, and returns the response.

 Hmm, try changing

while !eof(s) && nr < nb

to

while nr < nb && !eof(s)

in io.jl's readbytes! function, in case eof for the socket is blocking after the last byte is read.

[Olav Andreas Lindekleiv]

Yes, it works now. Thanks!

[Steven G. Johnson]

On Wednesday, August 14, 2013 4:17:41 AM UTC-4, Olav Andreas Lindekleiv wrote:

Yes, it works now. Thanks! 

The latest git version of Julia contains this fix.