WebSocket fragmentation
1,373 views
Skip to first unread message
Chris
Nov 11, 2011, 1:50:34 PM11/11/11
to Chromium HTML5
I'm writing WebSocket client and server applications for the HyBi-10
version of the protocol. I'm unable to solve fragmentation. I
believe I understand the eitf documentation but I seem to be missing
something.
When I run the server locally everything works great. When I run it
on an AWS EC2 instance I run into fragmentation problems. As per the
TCP protocol an intermediary seems to be fragmenting the message and I
can't put it back together properly.
For example, let's say Chrome sends a message to the server. It
decides to send it as 1 frame, not fragmented. The entire message is
3000 bytes. My server RECV's at 4096 bytes, but receives the message
(possibly others in between) in a series of chunks as small as 500
bytes. The intermediary breaking up the message is unaware of the
WebSocket protocol so the FIN bit is 1 even though several messages
are being sent to my server.
What do I do?
version of the protocol. I'm unable to solve fragmentation. I
believe I understand the eitf documentation but I seem to be missing
something.
When I run the server locally everything works great. When I run it
on an AWS EC2 instance I run into fragmentation problems. As per the
TCP protocol an intermediary seems to be fragmenting the message and I
can't put it back together properly.
For example, let's say Chrome sends a message to the server. It
decides to send it as 1 frame, not fragmented. The entire message is
3000 bytes. My server RECV's at 4096 bytes, but receives the message
(possibly others in between) in a series of chunks as small as 500
bytes. The intermediary breaking up the message is unaware of the
WebSocket protocol so the FIN bit is 1 even though several messages
are being sent to my server.
What do I do?
Fumitoshi Ukai (鵜飼文敏)
Nov 11, 2011, 2:10:15 PM11/11/11
to Chris, Chromium HTML5
I'm writing WebSocket client and server applications for the HyBi-10
version of the protocol. I'm unable to solve fragmentation. I
believe I understand the eitf documentation but I seem to be missing
something.
When I run the server locally everything works great. When I run it
on an AWS EC2 instance I run into fragmentation problems. As per the
TCP protocol an intermediary seems to be fragmenting the message and I
can't put it back together properly.
Why you can't put it back together?
A WebSocket frame is not bound to a TCP packet.
You should reconstruct WebSocket data frame from byte stream.
For example, let's say Chrome sends a message to the server. It
decides to send it as 1 frame, not fragmented. The entire message is
3000 bytes. My server RECV's at 4096 bytes, but receives the message
(possibly others in between) in a series of chunks as small as 500
bytes. The intermediary breaking up the message is unaware of the
WebSocket protocol so the FIN bit is 1 even though several messages
are being sent to my server.
What do I do?
You've read a WebSocket frame header, so you could know how many bytes would be in the frame.
Just read more.
--
ukai
--
You received this message because you are subscribed to the Google Groups "Chromium HTML5" group.
To post to this group, send email to chromiu...@chromium.org.
To unsubscribe from this group, send email to chromium-html...@chromium.org.
For more options, visit this group at http://groups.google.com/a/chromium.org/group/chromium-html5/?hl=en.
Chris
Nov 11, 2011, 2:27:36 PM11/11/11
to Chromium HTML5
The payload length does not specify the exact length of the message,
only the max number of payload data bytes are being used in order to
properly unmask the message. As well, unlike Hixie-76/HyBi-00 there
is no longer an EOM indicator.
I'll _try_ to illustrate this better...Inside my diagram a "." will
represent a piece of frame structure (FIN, RSV, Opcode, Payload len,
Masking-key) and a ":" will represent payload data. A "^" can
represent the end of a TCP fragment received by my server:
+-------------------^------+-------^----------------^---------^+
|..........::::::::::::::::::|...........::::::::::::::::::::::::::|
+-------------------^------+-------^----------------^---------^+
When my WebSocket RECV's data from the socket to unframe it doesn't
know if the previous message was at the end of it's payload data.
On Nov 11, 2:10 pm, Fumitoshi Ukai (鵜飼文敏) <u...@chromium.org> wrote:
> On Fri, Nov 11, 2011 at 10:50, Chris <cbo...@gmail.com> wrote:
> > I'm writing WebSocket client and server applications for the HyBi-10
> > version of the protocol. I'm unable to solve fragmentation. I
> > believe I understand the eitf documentation but I seem to be missing
> > something.
>
> > When I run the server locally everything works great. When I run it
> > on an AWS EC2 instance I run into fragmentation problems. As per the
> > TCP protocol an intermediary seems to be fragmenting the message and I
> > can't put it back together properly.
>
> Why you can't put it back together?
> A WebSocket frame is not bound to a TCP packet.
> You should reconstruct WebSocket data frame from byte stream.http://tools.ietf.org/html/draft-ietf-hybi-thewebsocketprotocol-17#se...
only the max number of payload data bytes are being used in order to
properly unmask the message. As well, unlike Hixie-76/HyBi-00 there
is no longer an EOM indicator.
I'll _try_ to illustrate this better...Inside my diagram a "." will
represent a piece of frame structure (FIN, RSV, Opcode, Payload len,
Masking-key) and a ":" will represent payload data. A "^" can
represent the end of a TCP fragment received by my server:
+-------------------^------+-------^----------------^---------^+
|..........::::::::::::::::::|...........::::::::::::::::::::::::::|
+-------------------^------+-------^----------------^---------^+
When my WebSocket RECV's data from the socket to unframe it doesn't
know if the previous message was at the end of it's payload data.
On Nov 11, 2:10 pm, Fumitoshi Ukai (鵜飼文敏) <u...@chromium.org> wrote:
> On Fri, Nov 11, 2011 at 10:50, Chris <cbo...@gmail.com> wrote:
> > I'm writing WebSocket client and server applications for the HyBi-10
> > version of the protocol. I'm unable to solve fragmentation. I
> > believe I understand the eitf documentation but I seem to be missing
> > something.
>
> > When I run the server locally everything works great. When I run it
> > on an AWS EC2 instance I run into fragmentation problems. As per the
> > TCP protocol an intermediary seems to be fragmenting the message and I
> > can't put it back together properly.
>
> Why you can't put it back together?
> A WebSocket frame is not bound to a TCP packet.
>
> > For example, let's say Chrome sends a message to the server. It
> > decides to send it as 1 frame, not fragmented. The entire message is
> > 3000 bytes. My server RECV's at 4096 bytes, but receives the message
> > (possibly others in between) in a series of chunks as small as 500
> > bytes. The intermediary breaking up the message is unaware of the
> > WebSocket protocol so the FIN bit is 1 even though several messages
> > are being sent to my server.
> > What do I do?
>
> You've read a WebSocket frame header, so you could know how many bytes
> would be in the frame.
> Just read more.
>
> --
> ukai
>
>
>
>
>
>
>
>
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Chromium HTML5" group.
> > To post to this group, send email to chromium-ht...@chromium.org.
> > For example, let's say Chrome sends a message to the server. It
> > decides to send it as 1 frame, not fragmented. The entire message is
> > 3000 bytes. My server RECV's at 4096 bytes, but receives the message
> > (possibly others in between) in a series of chunks as small as 500
> > bytes. The intermediary breaking up the message is unaware of the
> > WebSocket protocol so the FIN bit is 1 even though several messages
> > are being sent to my server.
> > What do I do?
>
> You've read a WebSocket frame header, so you could know how many bytes
> would be in the frame.
> Just read more.
>
> --
> ukai
>
>
>
>
>
>
>
>
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Chromium HTML5" group.
> > To unsubscribe from this group, send email to
> > chromium-html5+unsubscr...@chromium.org.
Fumitoshi Ukai (鵜飼文敏)
Nov 11, 2011, 2:32:35 PM11/11/11
to Chris, Chromium HTML5
The payload length does not specify the exact length of the message,
only the max number of payload data bytes are being used in order to
properly unmask the message. As well, unlike Hixie-76/HyBi-00 there
is no longer an EOM indicator.
No, it is the exact length of the message (most case, browser doesn't fragment the message in webscket level), or it would be fragmented in websocket level then you could check FIN bit in the websocket frame header, as end of message.
I'll _try_ to illustrate this better...Inside my diagram a "." will
represent a piece of frame structure (FIN, RSV, Opcode, Payload len,
Masking-key) and a ":" will represent payload data. A "^" can
represent the end of a TCP fragment received by my server:
Don't assume TCP packet is aligned to the WebSocket frame.
WebSocket uses TCP as byte stream.
--
ukai
To post to this group, send email to chromiu...@chromium.org.
To unsubscribe from this group, send email to chromium-html...@chromium.org.
Chris
Nov 11, 2011, 3:57:26 PM11/11/11
to Chromium HTML5
Thank you Fumitoshi. I previously misunderstood the payload len bytes
in the spec and confused it with protocol fragmenting where the number
of fragments is not known.
in the spec and confused it with protocol fragmenting where the number
of fragments is not known.
Reply all
Reply to author
Forward
0 new messages