How to transfer huge files using websocket++?

[Jerry Yin]

Hi Peter,

We used websocket 0.3 for more than half a year and everything looks good till now.

Now we are trying to implement a new feature to transfer a file as a message. I just load the files and read them to a buffer and send the whole buffer as a message. It works for small files, but for huge files like 1 GB, it will eat up a lot of memory. Are there existing solution for sending the huge files in chunk?

I read the link below, but I'm still not sure how to do this or if websocket support that.
https://github.com/zaphoyd/websocketpp/issues/160

Thanks and regards,
Jerry

[Peter Thorson]

To confirm, you are wanting to send a 1GB file in a single WebSocket message?

There are two strategies here.

1. Application layer chunking: Decide that you will chop any file larger than x bytes into chunks. Send an initial metadata message with your chunk sizes (or wrap with start/fin messages).

2. WebSocket layer fragmentation: The WebSocket protocol allows for sending messages too large to buffer (or of unknown size when the first byte is sent) by splitting the message into separate frames which are automatically reassembled on the other end.

Presently WebSocket++ supports receiving fragmented messages (but they are still fully buffered on the receiving end). It does not support sending fragmented messages (feature tracked here: https://github.com/zaphoyd/websocketpp/issues/111) or directly receiving fragments before the message is complete (feature tracked here: https://github.com/zaphoyd/websocketpp/issues/34). This is the same behavior of all modern web browsers, so if you are using a browser as a client anywhere you’ll likely run into this issue as well on that end.

I do hope to implement fragmented send at some point, as it will be useful for some of the WebSocket load testing software I am working on. I presently have little interest in a frame based receive API as I don’t believe that the benefits outweigh the additional complexity. As mentioned in issue #160, both of these features are invasive to implement and I feel like there are more important features and fixes outstanding at the moment. I am open to discussion this issue.

--
You received this message because you are subscribed to the Google Groups "WebSocket++" group.
To unsubscribe from this group and stop receiving emails from it, send an email to websocketpp...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.