Neko server. Problem with websockets when too much data.

[Horsetopus]

Hello Haxe wielders

So I have a project where I run a neko server, and connect to it, from my haxe build js clients, through websockets.

For the server side, I use the haxe-websocket library

Everything was going ok.

But now I am trying to send some files through these websockets.

I know, I know. I should use XMLHttpRequests. But it would require some extra setup whereas my websockets are already there. So I want to give it a try.

The problem is that if I send data too fast, a few of the requests work great, but then I end up with an error:

ServerLoop.hx:144: OutsideBounds

Called from /usr/share/haxe/std/haxe/io/Bytes.hx line 362

Called from neko/net/WebSocketServerLoop.hx line 118

Called from /usr/share/haxe/std/neko/net/ServerLoop.hx line 170

Called from /usr/share/haxe/std/neko/net/ServerLoop.hx line 218

If I send a string too big, I get another error, from the very first request:

ServerLoop.hx:144: Invalid char null at position 0

Called from D:\devtools\HaxeToolkit\haxe\std/haxe/format/JsonParser.hx line 251

Called from D:\devtools\HaxeToolkit\haxe\std/haxe/format/JsonParser.hx line 133

Called from the/test/code/Main.hx line 37

Called from neko/net/WebSocketServerLoop.hx line 97

Called from D:\devtools\HaxeToolkit\haxe\std/neko/net/ServerLoop.hx line 170

Called from D:\devtools\HaxeToolkit\haxe\std/neko/net/ServerLoop.hx line 218 

Now if I can't send files because it's too slow, ok I'll find another way.

But the problem is that it actually mess up the connection and close it, this makes me worried, not only for the file sending, but the whole process...

Is this behavior normal?

Can I really rely on websockets? How to catch this kind of errors? How to handle them?

I have tested on both Window and Linux, same issues.

Attached are the files of my windows tests (change FileUpload.CHARS_PER_SLICE to trigger the different type of issues ).

To run, launch the server, launch the client, upload a big file by drag and dropping. If it goes ok, try again, maybe with a bigger file, or upload multiple ones at once.

Thank you for your help.

[Horsetopus]

Here are the test files, Google group wouldn't let me post with them attached:

https://drive.google.com/file/d/0B0HEnYNo2baeclFvWXRBbVpvU3M

[Horsetopus]

Pwetty Pwease?
Anybody?

[Horsetopus]

You guys, I'm seriously!

Has anybody run my code?

Nobody has any insight?

[Cambiata]

Hi!

No experience with web sockets, but maybe you could try another server implementation?
For example https://github.com/soywiz/haxe-ws

I tried to run your code, but it doesn't seem to work localhost with nekoserver.
Maybe apache or equivalent is needed?

/ Jonas

[Rodislav Moldovan]

Hello, 

I'm using same library, everything works fine, still I was not sending files but I was spamming the app while stress testing with gatling by sending tons of messages, no issues so far.

but similar problems, as you described above, I saw when I've extended their function to parse data, the function is called: processClientData (here is what I did to make it work nicely)

override function processClientData(d:TClientData, buf:Bytes, bufpos:Int, buflen:Int) : Int

{

for (i in (bufpos)...buflen)

{

// http://help.adobe.com/en_US/ActionScript/3.0_ProgrammingAS3/WS5b3ccc516d4fbf351e63e3d118a9b90204-7cfb.html

// https://en.wikipedia.org/wiki/ASCII

// https://en.wikipedia.org/wiki/Null_character

if (buf.get(i) == 0x0)

{

var msg:String = buf.getString(bufpos, i - 1 - bufpos);

processIncomingMessage(d, buf.getString(bufpos, i - bufpos));

return i + 1 - bufpos;

}

}

return 0;

}

[Horsetopus]

Hi Cambiata,

Sorry for the delay, and thank you for your reply.

I considered trying other libraries, but it will be a last measure.

A lot of my code depend on this library.

Thankfully I think I fixed it.

[Horsetopus]

Hi,

Thank you for your answer and sorry for my late reply.

Indeed processClientData is where µI ended up tracking my issue.

But the 0x00 bit is never the one I get.

Is think maybe you set your websocket as binary? I send it as text ( my files are split in pieces, witch are Base64 encoded ).

For me its mostly 0x81, and sometimes ( and that's where the code would throw a mistake ) 0x01 and 0x80.

Turn out sometimes the websocket client can decide to split the message in two.

Obviously it appends a lot if you send big data ( files for example ).

And then it first sends a 0x01 message and then a 0x80 message.

So all I had to do is add a "temp" variable to ClientData. If it's a 0x01 message, I write to this "temp" variable, and it it's 0x80 I concatenate to "temp" and send the result.

So simple... and yet it took me a while.

At least I hope it will help someone.

I'll write to the coder so that he uploads his library.

Thanks anyway.