haxe.Http error when behind a proxy

[Tynril]

Hello,

When using haxelib (specifically, doing "haxelib upgrade") behind a proxy, I often get a "Invalid string length" for some libraries. Sometimes it is random, but it can be reproduced almost all the time with the nme library. I have started debugging a little bit, and it seems that the issue comes from the haxe.Http and isn't specific to haxelib. It is triggered in haxelib because it uses Haxe Remoting through an HttpConnection, which uses the haxe.Http class.

In haxelib's case (which is haxe.remoting.HttpConnection's case), the Http class is used this way:

var data = null;

var h = new haxe.Http(__url);

#if js

h.async = false;

#end

#if (neko && no_remoting_shutdown)

h.noShutdown = true;

#end

#if (neko || php || cpp)

h.cnxTimeout = TIMEOUT;

#end

var s = new haxe.Serializer();

s.serialize(__path);

s.serialize(params);

trace("Parameter: " + s.toString());

h.setHeader("X-Haxe-Remoting","1");

h.setParameter("__x",s.toString());

h.onData = function(d) { data = d; };

h.onError = function(e) { throw e; };

h.request(true);

At that point, "data" is supposed to be filled with the full remoting data, and is passed to the haxe.Unserializer subsystem before being returned. The issue is, behind that proxy (which is squid 3.1.15 btw), the data are truncated, which leads to an error during the unserialization. When using a REST debugger, behind the same proxy, I get the full data. I guess that the haxe.Http class has some buffer issue, or doesn't wait for the whole data to be recieved, or something like that. But I'm not knowledgeable enough when it comes to such low-level classes to suggest a fix.

Does anyone has an experience on that?

Thanks!

[Nicolas Cannasse]

Le 07/08/2012 18:14, Tynril a �crit :

That would be interesting if you could add some traces to haxe.Http neko
implementation in order to check what's happenning in that cases versus
when it's working.

Best,
Nicolas

[Tynril]

Hello again,

I've continued my investigation, and got it working.

The issue happens in the haxe.Http.readHttpResponse function. I get no "content-length" header from the proxy, so the size can't be set before reading the data. So I subsequently get inside the (size == null) case, where the socket is shutdown at first (because noShutdown is set to false), and the data are downloaded until an EOF error is raised by the socket. Here's the logs I get:

Http.hx:535: Trying to read from the socket...

Http.hx:537: Read 1024 bytes

Http.hx:535: Trying to read from the socket...

Http.hx:537: Read 1024 bytes

Http.hx:535: Trying to read from the socket...

Http.hx:537: Read 544 bytes

Http.hx:535: Trying to read from the socket...

Http.hx:546: Encountered an EOF error, stopped reading.

At this point, the data inside the buffer are truncated, and the unserialization fails.

I got it working by setting noShutdown to true. That changes the logs to:

Http.hx:535: Trying to read from the socket...

Http.hx:537: Read 1024 bytes

Http.hx:535: Trying to read from the socket...

Http.hx:537: Read 1024 bytes

Http.hx:535: Trying to read from the socket...

Http.hx:537: Read 544 bytes

Http.hx:535: Trying to read from the socket...

Http.hx:537: Read 1024 bytes

Http.hx:535: Trying to read from the socket...

Http.hx:537: Read 322 bytes

Http.hx:535: Trying to read from the socket...

Http.hx:546: Encountered an EOF error, stopped reading.

And at that point, the data were complete.

It's still hard to be sure that the issue really did came from that part, because the proxy caching tends to be quite random, but apparently, it did the trick. Now, by looking at the HttpConnection class of haxe.remoting, it seems that the noShutdown flag is set when compiled with the "no_remoting_shutdown" flag. Maybe it would be worth setting that flag when a proxy is used? I don't really understand why we'd like to shutdown the socket before having read its input anyway, but there's probably a good reason for that, right?

I hope that helped!

On 8 August 2012 11:27, Nicolas Cannasse <ncan...@motion-twin.com> wrote:

--
To post to this group haxe...@googlegroups.com
http://groups.google.com/group/haxelang?hl=en

[Tynril]

Also, if anyone wants that issue quick-fixed as well before Nicolas takes a look at it, here's the how-to:

 - Open the "Haxe/std/tools/haxelib/haxelib.hxml" file located where you installed Haxe;

 - Add "-D no_remoting_shutdown" (without the quotes) before the last line (the one that starts with "-cmd");

 - Open a terminal or a command line tool;

 - Navigate to the folder that contains this file;

 - Run "haxe haxelib.hxml";

 - You're done.

[Nicolas Cannasse]

Le 08/08/2012 14:56, Tynril a �crit :

> Hello again,
>
> I've continued my investigation, and got it working.
>
> The issue happens in the haxe.Http.readHttpResponse function. I get no
> "content-length" header from the proxy, so the size can't be set before
> reading the data. So I subsequently get inside the (size == null) case,
> where the socket is shutdown at first (because noShutdown is set to
> false), and the data are downloaded until an EOF error is raised by the
> socket. Here's the logs I get:

[...]

> It's still hard to be sure that the issue really did came from that
> part, because the proxy caching tends to be quite random, but
> apparently, it did the trick. Now, by looking at the HttpConnection
> class of haxe.remoting, it seems that the noShutdown flag is set when
> compiled with the "no_remoting_shutdown" flag. Maybe it would be worth
> setting that flag when a proxy is used? I don't really understand why
> we'd like to shutdown the socket before having read its input anyway,
> but there's probably a good reason for that, right?

Some HTTP servers have the bad habit to keep the connection opened until
the client closes it... In order to prevent such bad scenarios from
happening (and locking your call) we shutdown the socket for writing,
which mean that we inform the server that we will no longer send data,
but still accept to read some. Looks like your proxy wrongly assume we
are closing the connection here.

I would be happy if we can find a way to make it work perfectly
everywhere. Sadly I no longer have an URL that causes the original
shutdown to be added.

Best,
Nicolas

[Tynril]

What about putting the shutdown after all data were fetched? Wouldn't that do the trick for both cases?

On 8 August 2012 17:29, Nicolas Cannasse <ncan...@motion-twin.com> wrote:

Best,
Nicolas

[Nicolas Cannasse]

Le 08/08/2012 17:43, Tynril a �crit :

> What about putting the shutdown after all data were fetched? Wouldn't
> that do the trick for both cases?

Well, the whole problem is that unless you have Content-Length, you
don't know if all the data has been fetched or not...

Best,
Nicolas

[Tynril]

You're right, that's a mostly unsolvable issue. Now, it seems that it isn't directly related to the proxy, because squid usually forwards Content-Length headers correctly, and by using an external HTTP request viewer, I can see that http://lib.haxe.org/ does not send the Content-Length header at all. Also, when checking the headers of http://haxe.org/, it seems that no such header is sent either.

Maybe it could be interesting to fix it the other way, by ensuring that Content-Length is correctly sent by the Neko web layer?

On 8 August 2012 23:14, Nicolas Cannasse <ncan...@motion-twin.com> wrote:

Le 08/08/2012 17:43, Tynril a écrit :

What about putting the shutdown after all data were fetched? Wouldn't
that do the trick for both cases?

Well, the whole problem is that unless you have Content-Length, you don't know if all the data has been fetched or not...

Best,
Nicolas

[Tynril]

I have now more information regarding that issue.

The lack of Content-Length header on haxe.org websites (including lib.haxe.org) seems not to be directly connected to Haxe Remoting, but to mod_neko or Apache 1.x. The Content-Length header is properly set when calling Haxe Remoting on an Apache 2.x server using mod_neko2, but is lacking when using Apache 1.x with mod_neko.

I don't have much knowledge in Apache module development, but the differences between mod_neko and mod_neko2 seems to be very close, so maybe it could be easy to pinpoint the bug?

Anyway, the root of the problem is when using haxelib behind a proxy, so that could also be fixed quite easily by upgrading lib.haxe.org to Apache 2.x. I'm pretty sure it's not really safe to use Apache 1.x anymore, anyway.

What do you think?